Language models can classify text — tickets, emails, reviews, documents — with little or no training data.
Define Categories Well
For each category give a name, a clear definition, examples, and how it differs from neighbouring categories. Ambiguous categories produce inconsistent results.
Handle the Awkward Cases
- None fits: include an "other" category and describe when to use it.
- Several fit: say whether to choose one (the primary) or allow multiple labels.
- Unreadable or empty input: specify the output.
Constrain the Output
Ask for only the category name, from an exact list, or use structured output with an enumerated field. This makes results easy to parse and count.
Few-Shot Examples
A few labelled examples per category, including borderline ones, improve consistency.
Reasoning for Hard Cases
For nuanced categories, asking for a brief justification before the label can help — and makes errors easier to diagnose.
Evaluate Properly
Label a representative sample by hand and measure accuracy, per-class precision and recall, and the confusion matrix.
Cost at Scale
For high volumes, compare with a smaller model or a traditional classifier trained on LLM-labelled data. The LLM can bootstrap labels; a cheaper model can do the bulk work.