Comparison
Sampling vs Test-Time Compute
Sampling and Test-Time Compute are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Sampling
Sampling comes up when the question is fundamentally about inference.
OpenAI default: temperature 1.0, top-p 1.0.
When you would reach for Test-Time Compute
Whenever quality matters more than latency — math, code, research, structured planning.
o1 thinking for 30 seconds before answering a math olympiad problem.
Frequently asked
What is the difference between Sampling and Test-Time Compute?
Sampling: Sampling is the act of choosing the next token from the model's output distribution, typically after applying temperature and a truncation strategy like top-p or top-k. Test-Time Compute: Test-time compute is the extra reasoning, sampling, or search a model can do at inference time to improve quality — more thinking tokens, more candidate answers, or verifier-guided search.
When should I use Sampling vs Test-Time Compute?
Sampling is the right concept when you are focused on inference. Whenever quality matters more than latency — math, code, research, structured planning.
Are Sampling and Test-Time Compute the same thing?
No. Sampling is inference; Test-Time Compute is prompting. They are related but address different parts of the AI stack.