A chunk that reads "Revenue grew 3% over the previous quarter" is ambiguous on its own: which company, which quarter? Retrieval struggles to match it to a specific question.
The Problem
Chunking separates text from the context that gives it meaning: the document title, the section, the entity being discussed, the time period.
Simple Fixes
- Prefix each chunk with the document title and section headings before embedding.
- Include key metadata (date, product, region) in the indexed text.
Generated Context
A more thorough approach uses a language model to write a short description situating each chunk within its whole document — "This chunk is from Acme's Q2 2025 earnings report and discusses revenue growth compared with Q1" — and prepends it before embedding and keyword indexing.
Costs
Generating context for every chunk means a model call per chunk at indexing time. Prompt caching of the full document across calls for the same document can reduce the cost substantially.
Measure the Gain
Compare retrieval recall with and without added context on your test questions. Gains are typically largest for documents with many similar-looking passages, such as reports, contracts and manuals.
Keep the Original Text
Store the original chunk text separately so the language model sees the real content, with the added context kept clearly distinguishable.