Comparison
Fine-tuning vs Reinforcement Learning from Human Feedback
Fine-tuning and Reinforcement Learning from Human Feedback are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Fine-tuning
After you've exhausted prompting and retrieval, and you have a few hundred to thousands of clean labeled examples.
Fine-tuning Llama 3 on medical Q&A for a clinical assistant.
When you would reach for Reinforcement Learning from Human Feedback
Reinforcement Learning from Human Feedback comes up when the question is fundamentally about training.
ChatGPT trained with RLHF to refuse unsafe requests.
Frequently asked
What is the difference between Fine-tuning and Reinforcement Learning from Human Feedback?
Fine-tuning: Fine-tuning continues training a pretrained model on a smaller, task-specific dataset, adjusting its weights to specialize behavior or knowledge. Reinforcement Learning from Human Feedback: RLHF fine-tunes an LLM to maximize a reward model that was itself trained on human preference judgments between candidate responses.
When should I use Fine-tuning vs Reinforcement Learning from Human Feedback?
After you've exhausted prompting and retrieval, and you have a few hundred to thousands of clean labeled examples. Reinforcement Learning from Human Feedback applies when you are focused on training.
Are Fine-tuning and Reinforcement Learning from Human Feedback the same thing?
No. Fine-tuning is training; Reinforcement Learning from Human Feedback is training. They are related but address different parts of the AI stack.