Not every task needs the largest model. Small language models — from under a billion to a few billion parameters — have become surprisingly capable.
Advantages
- Cost: far cheaper per request.
- Speed: lower latency.
- Deployment: can run on laptops, phones or modest servers.
- Privacy: local deployment keeps data in-house.
Good Fits
- Classification, extraction and routing.
- Simple summarisation.
- Autocomplete and on-device features.
- High-volume pipelines.
- Specialised tasks after fine-tuning.
Limitations
- Weaker reasoning and knowledge.
- Less reliable instruction following on complex prompts.
- Smaller context windows in some cases.
Getting the Most From Them
- Keep tasks focused and prompts clear.
- Provide examples.
- Fine-tune on your task.
- Distil from larger models: train on a big model's outputs.
- Route: send easy requests to small models and hard ones to large models.
Evaluate
Test on your tasks. A small model that reaches acceptable quality can transform the economics of an application.