Comparison
Reward Model vs Reinforcement Learning from Human Feedback
Reward Model and Reinforcement Learning from Human Feedback are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Reward Model
Reward Model comes up when the question is fundamentally about training.
Anthropic's preference model trained on HH-RLHF data.
When you would reach for Reinforcement Learning from Human Feedback
Reinforcement Learning from Human Feedback comes up when the question is fundamentally about training.
ChatGPT trained with RLHF to refuse unsafe requests.
Frequently asked
What is the difference between Reward Model and Reinforcement Learning from Human Feedback?
Reward Model: A reward model scores model outputs the way humans would, learned from preference data. RLHF then optimizes the policy LLM to maximize the reward model's score. Reinforcement Learning from Human Feedback: RLHF fine-tunes an LLM to maximize a reward model that was itself trained on human preference judgments between candidate responses.
When should I use Reward Model vs Reinforcement Learning from Human Feedback?
Reward Model is the right concept when you are focused on training. Reinforcement Learning from Human Feedback applies when you are focused on training.
Are Reward Model and Reinforcement Learning from Human Feedback the same thing?
No. Reward Model is training; Reinforcement Learning from Human Feedback is training. They are related but address different parts of the AI stack.