RAG is powerful but not always the right answer.
The Information Is Small and Stable
If all the needed knowledge fits in a prompt and rarely changes, include it directly. Retrieval adds complexity for no gain.
The Question Is About Numbers in Tables
Aggregations and filters over structured data are better answered by querying the database, possibly with text-to-SQL.
Exact Lookups
Looking up an order status or an account balance calls for an API or database query, not semantic search over documents.
Behaviour, Not Knowledge
If the problem is tone, format or following a procedure, better prompting or fine-tuning is the fix.
The Content Isn't Ready
If documents are outdated, contradictory or missing, RAG will confidently surface the mess. Fix the content first — sometimes a well-organised FAQ page solves the problem without any AI.
Search Is What Users Want
Sometimes people want to find and read the document, not a generated answer. A good search experience may serve them better.
High-Stakes Answers Without Review
Legal, medical or financial determinations shouldn't be generated and delivered automatically, however good the retrieval.
Decide From the Problem
Start from the user's need and the nature of the information, then choose the simplest approach that meets it.