Comparison
Pairwise Comparison vs Preference Data
Pairwise Comparison and Preference Data are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Pairwise Comparison
Pairwise Comparison comes up when the question is fundamentally about evaluation.
Comparing prompt v3 vs prompt v4 on 200 fixed examples: GPT-4 judge picks v4 as better in 58% of cases (with 6% ties).
When you would reach for Preference Data
Preference Data comes up when the question is fundamentally about training.
Anthropic HH-RLHF (~170K preference pairs).
Frequently asked
What is the difference between Pairwise Comparison and Preference Data?
Pairwise Comparison: Pairwise comparison asks a judge — human or LLM — to pick the better of two responses to the same prompt. Aggregates to a win rate; the dominant method for comparing model or prompt versions. Preference Data: Preference data is collections of (chosen, rejected) response pairs over the same prompt. It is the fuel for DPO and reward-model training.
When should I use Pairwise Comparison vs Preference Data?
Pairwise Comparison is the right concept when you are focused on evaluation. Preference Data applies when you are focused on training.
Are Pairwise Comparison and Preference Data the same thing?
No. Pairwise Comparison is evaluation; Preference Data is training. They are related but address different parts of the AI stack.