Comparison
Preference Data vs Reinforcement Learning from Human Feedback
Preference Data and Reinforcement Learning from Human Feedback are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Preference Data
Preference Data comes up when the question is fundamentally about training.
Anthropic HH-RLHF (~170K preference pairs).
When you would reach for Reinforcement Learning from Human Feedback
Reinforcement Learning from Human Feedback comes up when the question is fundamentally about training.
ChatGPT trained with RLHF to refuse unsafe requests.
Frequently asked
What is the difference between Preference Data and Reinforcement Learning from Human Feedback?
Preference Data: Preference data is collections of (chosen, rejected) response pairs over the same prompt. It is the fuel for DPO and reward-model training. Reinforcement Learning from Human Feedback: RLHF fine-tunes an LLM to maximize a reward model that was itself trained on human preference judgments between candidate responses.
When should I use Preference Data vs Reinforcement Learning from Human Feedback?
Preference Data is the right concept when you are focused on training. Reinforcement Learning from Human Feedback applies when you are focused on training.
Are Preference Data and Reinforcement Learning from Human Feedback the same thing?
No. Preference Data is training; Reinforcement Learning from Human Feedback is training. They are related but address different parts of the AI stack.