Summarisation is one of the most common and valuable uses of language models, and one of the easiest to get subtly wrong.
Say Who the Summary Is For
A summary for an executive, an engineer and a customer should look different. State the audience, the purpose and what decisions it will inform.
Specify Length and Shape
"Five bullet points of no more than 20 words each" is clearer than "a short summary". Ask for sections if the reader needs structure: key points, decisions, risks, next steps.
Say What Matters
Tell the model what to prioritise — figures, deadlines, commitments, disagreements — and what to leave out.
Guard Against Invented Details
- Instruct the model to use only information in the source.
- Ask it to quote or reference the passage behind each key point.
- Ask it to flag uncertainty or contradictions in the source rather than resolve them silently.
Long Documents
For documents longer than the context window, summarise section by section and then summarise the summaries, keeping key figures verbatim.
Checking Summaries
- Verify every number, name and date against the source.
- Check for omissions: did it miss a major point or caveat?
- For recurring summarisation tasks, build a small test set and grade faithfulness and coverage.
Watch for Tone Drift
Models can make summaries sound more certain or more positive than the original. Ask it to preserve hedges and qualifications.