RAG Architecture: The Components End to End

A retrieval-augmented generation (RAG) system has more moving parts than "search plus a language model". Knowing each one makes problems easier to locate.

Offline: Preparing Knowledge

1. Sources — documents, wikis, tickets, databases.
2. Parsing — turning PDFs, HTML and slides into clean text with structure.
3. Chunking — splitting text into retrievable passages.
4. Enrichment — titles, headings, dates, permissions and other metadata.
5. Indexing — embeddings in a vector index, plus a keyword index.

Online: Answering a Question

1. Query understanding — rewriting, expanding or classifying the question.
2. Retrieval — finding candidate chunks with semantic and keyword search, filtered by metadata and permissions.
3. Re-ranking — ordering candidates precisely.
4. Context assembly — choosing chunks and formatting them for the prompt.
5. Generation — the model answers from the context, with citations.
6. Post-processing — validation, citation checks and guardrails.

Around It

• Evaluation of retrieval and answers on a test set.
• Monitoring of quality, latency and cost.
• Feedback from users to find failures.
• Refresh pipelines keeping the index current.

Where Problems Usually Are

Most poor RAG answers trace back to parsing, chunking or retrieval — not the language model. Measure each stage separately.