Modern language models go through several training stages.
Pre-Training
The model learns to predict the next token across vast amounts of text and code. This teaches grammar, facts, reasoning patterns and coding skills. It's by far the most expensive stage.
Supervised Fine-Tuning
The pre-trained model is trained on examples of instructions and good responses, teaching it to follow requests and hold conversations.
Learning From Feedback
Models are refined using preference data — people or AI systems rating which responses are better — via techniques such as reinforcement learning from human feedback (RLHF) or direct preference optimisation. Training against written principles is another approach.
Reinforcement Learning on Verifiable Tasks
Rewarding correct answers on tasks with checkable outcomes — maths, code with tests — improves reasoning.
Safety Training
Teaching models to refuse harmful requests, resist manipulation and be honest about uncertainty.
Why It Matters to Users
- Knowledge stops at a training cut-off.
- Behaviour reflects training choices, so models from different providers differ in style and strengths.
- Fine-tuning builds on these stages rather than replacing them.