Skip to content

Chunk Size and Overlap Tuning

How to choose chunk size and overlap for retrieval by testing against real questions rather than guessing.

Editorial team 2 min read

Chunk size is one of the most influential settings in a RAG system, and there's no universal right answer.

The Trade-off

  • Small chunks are precise: the retrieved text is focused. But they may lack context — a sentence saying "this limit applies" is useless without the limit.
  • Large chunks carry context but blur the embedding and fill the prompt with irrelevant text.

Overlap

Overlapping consecutive chunks by a fraction (for example 10–20%) helps when an answer spans a boundary. Too much overlap wastes storage and returns near-duplicate results.

Structure Beats Fixed Sizes

Splitting at headings, paragraphs and list boundaries usually outperforms splitting every N characters. Use a size limit as a ceiling, not the main rule.

Test Systematically

  1. Build a set of realistic questions with the passages that answer them.
  2. Index the same documents with several chunking settings.
  3. Measure recall@k for each.
  4. Check end-to-end answer quality for the best candidates.

Different Content, Different Settings

FAQs suit one chunk per question. Long policies suit section-sized chunks. Code suits function-level chunks. It's fine to use different strategies for different sources.

Parent–Child Retrieval

Retrieve small, precise chunks, but pass their larger parent section to the model. This combines precise matching with enough context to answer.

More in RAG

All RAG guides →
RAG Guide · 1 min

RAG Architecture: The Components End to End

A map of a complete retrieval-augmented generation system, from ingestion to answer, and what each component is responsible for.

RAG 1 min read 6 Dec 2025

RAG Guide · 2 min

Document Parsing for RAG

Turning PDFs, slides, HTML and scans into clean, structured text — the unglamorous step that decides RAG quality.

RAG 2 min read 5 Dec 2025

RAG Guide · 2 min

Hybrid Search for RAG

Combining keyword and vector search so RAG finds both exact terms and paraphrased meaning.

RAG 2 min read 3 Dec 2025

RAG Guide · 1 min

Query Rewriting and Expansion

Improving retrieval by rewriting user questions: expansion, decomposition, hypothetical answers and conversational context.

RAG 1 min read 2 Dec 2025