Comparison
Chunking vs Retrieval-Augmented Generation
Chunking and Retrieval-Augmented Generation are both common AI/LLM terms but cover different ideas. Here is a quick side-by-side.
When you would reach for Chunking
Always — chunking is upstream of every other RAG decision. Spending 2 hours on chunking strategy commonly beats 2 weeks of prompt tuning.
A 50-page PDF split into 200-token chunks with 50-token overlap → ~150 chunks indexed.
When you would reach for Retrieval-Augmented Generation
When the model needs information that is not baked into its weights — fresh, private, or domain-specific.
"Chat with your PDFs" — Notion, Glean, ChatGPT custom GPTs.
Frequently asked
What is the difference between Chunking and Retrieval-Augmented Generation?
Chunking: Chunking is the process of splitting source documents into smaller passages before embedding them for retrieval. Chunk size and boundaries control how relevant retrievals will be. Retrieval-Augmented Generation: RAG retrieves relevant documents from a corpus at query time and includes them in the prompt, letting an LLM answer with up-to-date, source-cited, private information without retraining.
When should I use Chunking vs Retrieval-Augmented Generation?
Always — chunking is upstream of every other RAG decision. Spending 2 hours on chunking strategy commonly beats 2 weeks of prompt tuning. When the model needs information that is not baked into its weights — fresh, private, or domain-specific.
Are Chunking and Retrieval-Augmented Generation the same thing?
No. Chunking is agents & tools; Retrieval-Augmented Generation is agents & tools. They are related but address different parts of the AI stack.