Vector search finds passages with similar meaning; keyword search finds exact terms. RAG systems usually need both.
Why Vector Search Alone Falls Short
Embeddings handle paraphrase well but can miss exact product codes, error numbers, names, acronyms and rare technical terms. A question about "error E-4012" may retrieve passages about errors generally.
Why Keyword Search Alone Falls Short
It misses synonyms and natural-language phrasing: "can't sign in" won't match "authentication failure".
Combining Them
Run both searches and merge the result lists. Reciprocal rank fusion (RRF) is a simple, robust method: each document scores based on its rank in each list, so documents ranked well by both rise to the top. Weighted score combinations are an alternative but need score normalisation.
Tuning
- Retrieve more candidates from each method than you'll finally use.
- Adjust the balance based on your content: more keyword weight for technical documentation with codes, more semantic weight for conversational questions.
- Add a re-ranker after fusion for best precision.
Implementation
Many search engines and vector databases support hybrid search natively; otherwise run both and fuse the lists in application code.
Evaluate
Compare recall@k for keyword-only, vector-only and hybrid on your real questions. Hybrid usually wins, but measure it.