In conversation, users ask follow-ups that only make sense with history: "And for contractors?" or "What's the deadline for that?"
The Problem
Searching with the raw follow-up retrieves irrelevant results, because the question lacks the subject.
Rewrite Into a Standalone Query
Before retrieval, use a model to rewrite the latest message into a self-contained question using the conversation history:
"And for contractors?" → "How much annual leave do contractors get?"
Search with the rewritten question; answer the user's actual message.
Carrying Context
Include a short history in the generation prompt so answers stay coherent, but keep it trimmed — summarise older turns.
Re-Retrieve Each Turn
Don't assume earlier retrieved documents still apply; topics shift. Retrieve fresh context for each question, optionally keeping previously cited documents as candidates.
Detect Topic Changes
If the user moves to a new subject, don't let old context bias retrieval.
Clarifying Questions
When a question is ambiguous, asking a brief clarifying question can beat guessing.
Evaluate Multi-Turn Scenarios
Test sets should include conversations, not just single questions, checking that follow-ups are understood and answered correctly.