Most chat model APIs separate a system prompt — standing instructions for the whole conversation — from the user's messages.
What Belongs in a System Prompt
- The assistant's role and purpose.
- Audience and tone.
- Rules and boundaries: what it should and shouldn't do, and how to handle out-of-scope requests.
- Output conventions: format, length, citation style.
- Stable background information the assistant always needs.
Writing Good System Prompts
- Be specific: "Answer in plain English for small-business owners, under 150 words" beats "be helpful".
- Explain the reasons behind rules so the model can generalise sensibly.
- Describe what to do in tricky cases: missing information, requests outside scope, sensitive topics.
- Keep it organised with headings or tags; long unstructured prompts are hard for people and models alike.
What System Prompts Can't Do
They are guidance, not security controls. A determined user may get a model to deviate, and content the model reads can contain injected instructions. Enforce permissions and critical rules in your code, not only in the prompt.
Keep Them Versioned
Treat system prompts like code: store them in version control, review changes, and re-run your evaluation set whenever you change them.
Mind Cost
The system prompt is sent with every request. Keep it as short as it can be while still complete, and put it first so providers that support prompt caching can reuse it.