AI engineering / Beginner / 01.08
System Prompting
The standing instructions behind every chat: what goes in a system prompt, why products need one, and where it falls short.
14 min readFree
Standing instructions for every chat
The last lesson, Prompt Engineering, ended with a promise: next, system prompts, the standing instructions behind every chat. Here they are.
When you chat with an assistant inside a shop or a bank app, you only see your own message and its reply. But something else was sent first. A block of text the company wrote, that you never see. It tells the assistant who it is, how to talk and what to avoid. That text is the system prompt.
By the end you will know where it sits in a request, what goes into one, why products need it, and where it falls short. You will read a template for a support assistant and run three short programs. None of them calls a real model.
Key words
- Message
- One piece of a conversation sent to a model. It has a role and some text.
- Role
- A label on each message that says who is speaking: the system, the user or the assistant.
- System prompt
- Standing instructions the product's builder writes. They are sent with every request, apart from what the user types.
- User message
- What the person using the product types.
- Assistant message
- A reply written by the model.
- Brand voice
- The way a company sounds: calm, playful, formal, short.
- Prompt injection
- A trick where someone writes text meant to make a model ignore its instructions. It gets its own lesson later.
Where the system prompt sits
A chat with a model is a list of messages. Each message has a role. The OpenAI guide describes three of them: "developer messages are instructions provided by the application developer, prioritized ahead of user messages. user messages are instructions provided by an end user, prioritized behind developer messages. Messages generated by the model have the assistant role."
Names differ between services. OpenAI's guide says developer for the builder's messages, and also has a separate instructions setting. The Claude docs use a setting called system. Many tools still call the builder's text the system prompt, and so does this lesson. Think of it as one idea with a few names.
The OpenAI guide gives a handy picture. Developer and user messages are like a function and its arguments in programming: "developer messages provide the system's rules and business logic, like a function definition. user messages provide inputs and configuration to which the developer message instructions are applied, like arguments to a function."
A daily-life version: a restaurant. The owner writes the house rules once and pins them in the kitchen. Each customer then walks in with a different order. The rules stay the same. The orders change.
- 1
Builder writes the system prompt
Once, ahead of time. It is part of the product.
- 2
User types a message
Different every time.
- 3
The app joins them
It puts the system message first and the user message after, in one list.
- 4
The model replies
It reads the whole list and writes the assistant message.
The same OpenAI page says these instructions come "with differing levels of authority", and that any instructions given through its instructions setting "will take priority over a prompt in the input parameter". So the builder's words are meant to count for more than the user's.
Building the message list
Here is that list in plain Python. One helper, build_messages, puts a system message and a user message together. We call it for two different products with the same user question. The helper never changes. Only the system prompt does.
Each system prompt ends with two fixed rules, a tone and a word limit. We will use those in a minute. The code builds lists only. It calls no model.
def build_messages(system_prompt, user_text):
return [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_text},
]
bakery = "You are Pip, a cheerful helper for Crumb Bakery. Tone: friendly. Max words: 12."
bank = "You are the assistant for Northfield Bank. Tone: formal. Max words: 25."
question = "How do I change my order?"
for name, system_prompt in [("bakery", bakery), ("bank", bank)]:
messages = build_messages(system_prompt, question)
print(name, "->", len(messages), "messages")
for m in messages:
print(" ", m["role"] + ":", m["content"])Output
bakery -> 2 messages system: You are Pip, a cheerful helper for Crumb Bakery. Tone: friendly. Max words: 12. user: How do I change my order? bank -> 2 messages system: You are the assistant for Northfield Bank. Tone: formal. Max words: 25. user: How do I change my order?
Read the output. Both products got a list of 2 messages, and the user line is identical: "How do I change my order?". Only the system line differs. That small difference decides how each product will sound.
Quick check
In a request, which message is the same for every user of the product?
What goes in a system prompt
A good system prompt has a few parts. You do not need all of them every time, but these four cover most products.
Role and behaviour
Who the assistant is, who it talks to, and how it sounds.
Rules and limits
What it may do, and what it must never do.
Output format
How long a reply is, and how it is laid out.
When unsure
What to do instead of guessing.
Role and behaviour
Start with who the assistant is. The Claude docs say that "Setting a role in the system prompt focuses Claude's behavior and tone for your use case. Even a single sentence makes a difference". Their own sample is one line: a helpful coding assistant that specialises in Python.
Behaviour has three easy parts. Who is it, such as the support helper for a bakery? Who is it talking to, such as customers in a hurry? And how should it sound, such as warm, calm and plain? The OpenAI guide lists the same kind of thing: its instructions setting covers "tone, goals, and examples of correct responses".
Prompt Engineering showed a role as the first line of a template. In a system prompt it is the same idea, but it stands for the whole product, not for one question.
Rules and limits
Rules say what the assistant may and may not do. Write each one as a short, separate line. A product might limit the assistant to its own topics, forbid promises it cannot keep, or ask it to hand a hard question to a person.
Say what to do, not only what to avoid. "Never promise a refund" is good. "Say a team member will confirm refunds" is better, because it tells the model what to say instead.
Output format
Tell the assistant how long a reply should be and how it should look. A phone screen wants two or three short sentences. A report wants headings. The Prompt Engineering lesson showed that a format line changes replies, and the same is true here. The difference is that in a system prompt you write the format once and every reply follows it.
What to do when unsure
A model that does not know often still answers, and the answer can sound sure. Give it a safer path. Tell it to say it is not sure, and to pass the question to a person or ask the user for more detail. This one line prevents many wrong answers in a product that customers trust.
Why products need one
Without a system prompt, every reply would depend on what each user happened to type. With one, the product behaves the same way for everyone.
Same behaviour for every user
The builder writes the rules once. A new user and the ten-thousandth user are treated the same way.
Brand voice
A bakery sounds like a bakery and a bank sounds like a bank, even when asked the same question.
Safety rules
The system prompt is the place to state what the assistant must refuse or hand off. It is one layer of safety, not the only one.
This is why the OpenAI guide frames the developer message as the system's rules and business logic. The rules belong to the product, not to the user who is chatting.
Same question, two answers
Now we need something to answer. A real model needs a paid account and a network, so the code below uses a stand-in: a small function named fake_model. It looks in the system prompt for the line "Tone: friendly" or "Tone: formal", and for "Max words: N". Then it returns one of two fixed replies, cut to N words.
Be clear about what this is. The stand-in is not a real model. It follows two fixed rules so the code runs free. A real model is slower, costs money for every token it reads and writes, can give a different answer each time you ask, and can fail or refuse. A real model also reads the meaning of your instructions. This one only looks for two exact phrases.
def build_messages(system_prompt, user_text):
return [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_text},
]
def fake_model(messages):
"""A stand-in, not a real model. It follows two fixed rules it finds in the system prompt."""
system_prompt = messages[0]["content"]
tone = "friendly" if "Tone: friendly" in system_prompt else "formal"
limit = int(system_prompt.split("Max words: ")[1].rstrip("."))
if tone == "friendly":
reply = "Happy to help! Open Orders, tap Edit, and pick a new time."
else:
reply = "You may amend your order by signing in, selecting Orders, and choosing the Amend option before the cut-off time."
return " ".join(reply.split()[:limit])
bakery = "You are Pip, a cheerful helper for Crumb Bakery. Tone: friendly. Max words: 12."
bank = "You are the assistant for Northfield Bank. Tone: formal. Max words: 25."
question = "How do I change my order?"
for name, system_prompt in [("bakery", bakery), ("bank", bank)]:
answer = fake_model(build_messages(system_prompt, question))
print(name + ":", answer)
print(" words:", len(answer.split()))Output
bakery: Happy to help! Open Orders, tap Edit, and pick a new time. words: 12 bank: You may amend your order by signing in, selecting Orders, and choosing the Amend option before the cut-off time. words: 19
Same question, two answers. The bakery reply is friendly and 12 words long, which is its limit. The bank reply is formal and 19 words long, under its limit of 25. Nothing changed except the system prompt, which is the point of this lesson.
Tip
A stand-in shows how the pieces connect. It cannot show how a real model behaves. Always test your system prompt against the real model you plan to use.
A template for a support assistant
Here is a system prompt for a small bakery's support assistant. It is plain text, not run. It has the four parts from earlier: role, rules, format and what to do when unsure. At the end it holds a few notes the assistant may use.
You are Sol, the support assistant for Crumb Bakery.
Audience: customers who are in a hurry, often on a phone.
Tone: warm and calm. Plain words. No jokes about complaints.
Rules:
- Answer only about orders, opening hours and allergies.
- Never promise a refund. Say a team member will confirm it.
- Never guess an allergy fact. If it is not in the notes below, say so.
Format: at most 3 short sentences. No lists. No greeting.
When unsure: say "I am not sure", then offer to pass the question to a team member.
Notes:
- Open Tuesday to Sunday, 7am to 4pm.
- Orders can be changed up to 24 hours before pickup.Every line has a job. The audience line tells the model who is reading. The rules are short and each does one thing. The unsure line gives a safe exit. The notes hold the facts, so the model does not have to invent opening hours.
- 1
Write the role
One or two lines: who it is and who it serves.
- 2
List the rules
One rule per line. Say what to do, not only what to avoid.
- 3
Fix the format
Length, layout, and what to leave out.
- 4
Add the unsure path
What to say, and who to pass the question to.
- 5
Add the facts
Opening hours, policies, anything it must not guess.
- 6
Test it
Try ten real questions, odd ones too, and read every reply.
What a system prompt cannot do
A system prompt is strong, but it is still text a model reads. It has three limits.
People can push back
A user can try to talk the model out of its instructions. The Wikipedia page on prompt injection explains that the attack "takes advantage of the model's inability to distinguish between developer-defined prompts and user inputs". It also says that "relying solely on a system prompt crafted with instructions to be careful of injection attempts has limited effectiveness". Prompt injection gets its own lesson later.
It costs tokens every call
The system prompt is part of what the model reads each time, and you pay by the token. The Tokens lesson explains the unit. A long system prompt adds to every single call.
Long lists get followed unevenly
The more rules you stack, the more the model has to juggle, and it may follow some lines better than others. The OpenAI guide says model output is non-deterministic, so the only way to know is to test.
Next, a made-up sum for the second limit. Every number in the code below is made up, including the price. They only show the shape of the sum.
# Every number here is made up, only to show the shape of the sum.
system_prompt_tokens = 600
user_message_tokens = 40
calls_per_day = 10_000
made_up_price_per_1000_tokens = 0.002 # made up, not a real price
extra_tokens = system_prompt_tokens * calls_per_day
extra_cost = extra_tokens / 1000 * made_up_price_per_1000_tokens
print("system prompt tokens sent per day:", extra_tokens)
print("made-up cost of just the system prompt:", round(extra_cost, 2))
print("share of each call that is the system prompt:", round(system_prompt_tokens / (system_prompt_tokens + user_message_tokens) * 100), "%")Output
system prompt tokens sent per day: 6000000 made-up cost of just the system prompt: 12.0 share of each call that is the system prompt: 94 %
A 600-token system prompt sent 10,000 times a day is 6,000,000 tokens a day, and it would cost 12.0 at our made-up price. And of each call in this example, 94 % is the system prompt and only the rest is the user's message. Short, clear prompts cost less and are easier to follow. Real prices differ and change, so check the current pricing page of the service you use.
A note on secrets. Some products keep their system prompt secret. The Wikipedia page describes prompt leaking: a user using a chat prompt to reveal the software's system prompt, "something that is typically kept secret". So never put passwords, keys or private data in a system prompt, and do not rely on it staying hidden.
Tip
Treat the system prompt as a strong request, not a lock. Put anything that must never go wrong, such as payments or deletes, behind code and a person, as well as behind the words.
Quick check
Which line does NOT belong in a system prompt?
Check yourself
Quick check
Why does a product use a system prompt?
What to take with you
Four things to remember. The system prompt is the builder's standing text, apart from the user's message. It holds the role, the rules, the format and the unsure path. It gives every user the same behaviour. And it has limits: it can be pushed on, it costs tokens each call, and long lists get uneven attention.
Next: zero-shot and few-shot prompting, asking with and without examples.
Sources
Last updated on 16 November 2026
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.