Inference · beginner
Token Count
Token count is the number of tokens in a piece of text under a specific tokenizer. The unit of LLM pricing, context limits, and rate limits.
Explanation
Every prompt and completion is measured in tokens, not characters or words. English text averages ~4 chars per token in GPT-style tokenizers; code and non-Latin scripts can use noticeably more.
Knowing the token count of your prompt + expected output is the only way to estimate cost and stay within the context window. Providers expose token-counting endpoints or tokenizer libraries; the ModelTerms /tokenizer page does this client-side.
Examples
- "Hello, world!" = ~4 tokens (GPT-4o).
- A 1,000-word essay = ~1,300 tokens.
- A 200-page book = ~80K tokens.
Frequently asked
What is Token Count?
Token count is the number of tokens in a piece of text under a specific tokenizer. The unit of LLM pricing, context limits, and rate limits.
What is an example of token count?
"Hello, world!" = ~4 tokens (GPT-4o).
How is Token Count related to Token?
Token Count and Token are both inference concepts. A token is the basic unit an LLM reads and writes — usually a word piece (3-4 characters). LLMs are priced and sized by tokens, not words.
Is Token Count considered beginner?
Token Count is generally considered beginner-level material in the AI and LLM space.