Comparison
Sampling vs Streaming (LLM Responses)
Sampling and Streaming (LLM Responses) are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Sampling
Sampling comes up when the question is fundamentally about inference.
OpenAI default: temperature 1.0, top-p 1.0.
When you would reach for Streaming (LLM Responses)
Streaming (LLM Responses) comes up when the question is fundamentally about inference.
A ChatGPT-style web app: SSE stream rendering tokens as they arrive, TTFT ~0.6s vs full-wait of ~5s.
Frequently asked
What is the difference between Sampling and Streaming (LLM Responses)?
Sampling: Sampling is the act of choosing the next token from the model's output distribution, typically after applying temperature and a truncation strategy like top-p or top-k. Streaming (LLM Responses): Streaming returns tokens to the client as they're generated rather than holding the full response until completion. Implemented over Server-Sent Events (SSE) or WebSocket; what makes chat UIs feel fast.
When should I use Sampling vs Streaming (LLM Responses)?
Sampling is the right concept when you are focused on inference. Streaming (LLM Responses) applies when you are focused on inference.
Are Sampling and Streaming (LLM Responses) the same thing?
No. Sampling is inference; Streaming (LLM Responses) is inference. They are related but address different parts of the AI stack.