Many questions are about numbers in tables — sales by region, open tickets this week — which text-chunk retrieval handles poorly.
Why Chunks Struggle
Aggregations, filters and calculations require the whole dataset, not a few retrieved rows. A model reading ten chunks of a table can't reliably sum a column of ten thousand rows.
Text-to-SQL
A language model translates the question into a SQL query, which runs against the database; the model then explains the result.
To make it reliable:
- give the model the schema with clear table and column descriptions;
- include example questions and correct queries;
- restrict it to read-only access on specific tables or views;
- validate queries before running them and limit result sizes;
- show the query and result, so users can check.
Semantic Layers
Defining metrics and dimensions once ("revenue", "active customer") and having the model query that layer avoids inconsistent calculations.
Hybrid Questions
Some questions need both documents and data: "Why did churn rise in March?" may combine a churn query with retrieved release notes. Route or combine tools accordingly.
Evaluation
Build a set of questions with correct answers and check query results, not just query text — different SQL can be equally correct.
Security
Never let generated SQL run with write permissions or access to sensitive tables the user can't otherwise see.