esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

AI engineering / Beginner / 01.33

Fine-tuning

Teach a model your style, format or narrow task by training it further on your own examples. When to try it, and what it costs.

13 min readFree

Changing the model, not the prompt

The last lesson, RAG & Dynamic Filters, ended with a promise: next, fine-tuning, changing the model itself instead of the prompt. Here it is.

Until now, every trick in this course changed what you send to the model: a better instruction, a few examples, extra facts. The model stayed as it was. Fine-tuning changes the model.

Think of a new hire who already knows how to write and talk to people. You hand them a folder of your best past replies and say: answer like these. Soon they answer in your house style without being told each time. That is the idea.

You will learn what fine-tuning is, what training data looks like, when to try it and what it costs. You will also build and check a small training file in Python. No model is trained here.

Key words

Pre-trained model
A model that has already learned from a huge amount of general text. It is your starting point.
Fine-tuning
Training a pre-trained model a bit more, on your own examples, so its default behaviour changes.
Training example
One sample of what you want: an input, and the ideal output for it.
JSON Lines (JSONL)
A text file with one complete JSON record on every line. The usual way to store training examples.
Validation set
A few examples you hold back and do not train on. You use them afterwards to test the result.
Parameters
The numbers inside a model that it learned. Training changes these numbers.
LoRA
A cheaper way to fine-tune that trains only a small add-on instead of every parameter.
Eval
A fixed set of test questions and a way to score the answers, so you can tell if a change helped.

What fine-tuning is

Wikipedia's page on fine-tuning in deep learning puts it this way: it is "the process of adapting a computational model trained for one task (the upstream task) to perform a different, usually more specific, task (the downstream task)".

The page also says how it works: fine-tuning "involves applying additional training (e.g., on new data) to the parameters of a neural network that have been pre-trained". In plain words, you take a model that was trained once, and you train it a little more on your data. Wikipedia adds that it is typically done with supervised learning, meaning examples that come with the right answer.

From a builder's side, that gives you a simple picture.

  1. 1

    Start

    Pick a pre-trained model that is already good at general language.

  2. 2

    Show

    Give it many examples of an input and the ideal output.

  3. 3

    Train

    A training run nudges the model's parameters toward producing those outputs.

  4. 4

    Use

    You get a new, customised model. Its default behaviour has moved toward your examples.

The change is in the model, not in your prompt. The style now lives inside it.

How a model learns from examples is taught in Machine Learning Foundations. Here we stay with the builder's question: should I do it?

Teaching behaviour, not facts

Fine-tuning changes how a model acts: style, tone, a fixed output format, or one narrow task done the same way every time.

The OpenAI documentation on supervised fine-tuning says it lets you train a model "with examples for your specific use case", and that "the result is a customized model that more reliably produces your desired style and content". Its list of fitting tasks reads like a list of behaviours: classification, nuanced translation, generating content in a specific format, and correcting instruction-following failures. The same docs say fine-tuning "can also enable a model to consistently format responses in a certain way or handle novel inputs".

Facts are treated differently. For information from outside the model's training, the advice is to put it in the prompt: "include text or image content that the model will need to generate a response from outside its training data. This could include data from private databases or current, up-to-the-minute information." That is what the RAG lesson showed.

A rule of thumb follows. "It does not know our latest price list" is a context problem. "It knows enough but always answers in the wrong shape" is a fit for fine-tuning.

Tip

Ask first: is the model missing information, or behaving wrongly? Only the second is a candidate for fine-tuning.

One risk: Wikipedia notes that fine-tuning "can degrade a model's robustness to distribution shifts". In plain words, a tuned model can get worse on inputs that look different from your examples. So test on examples you did not train on.

What training data looks like

Supervised fine-tuning data is a pile of pairs: an input, and the ideal output you wish the model had given. The OpenAI docs call them "example prompts and desired model output" and say they often use "human-generated 'ground truth' responses", the answers a careful person agrees are right.

Those pairs are usually stored as JSON Lines. The jsonlines.org page says the format has three requirements: UTF-8 encoding, each line is a valid JSON value, and the line terminator is a newline character. It is "a convenient format for storing structured data that may be processed one record at a time". A blank line is not a valid value.

For chat models, the OpenAI docs ask for JSONL "with one complete JSON structure on every line", in the chat format: a list of messages, each with a role and some content. A typical line has a system message (standing instructions), a user message (the input) and an assistant message (the ideal reply).

Some numbers from those docs help you plan. The minimum there is 10 examples, but "we see improvements from fine-tuning on 50–100 examples, but the right number for you varies greatly". They recommend starting with 50 well-crafted demonstrations. And they give a warning that saves time: "If 50 examples have no impact, rethink your task or prompt before adding training data."

A good example, the docs say, is as realistic as possible: the prompts and outputs you expect in your application, with specific, clear questions and answers.

Tip

Check every example before you train. A model is trained toward whatever you show it, mistakes included.

Cheaper tuning with LoRA

Retraining every parameter of a huge model is costly. Some methods train only a small part. A well-known one is LoRA.

The LoRA paper's abstract explains the problem and the fix. Full fine-tuning "retrains all model parameters" and becomes less feasible as models grow. LoRA instead "freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer", which greatly reduces the number of trainable parameters. Frozen means not changed. Compared to GPT-3 175B fine-tuned with Adam, the authors report that it can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times.

Wikipedia describes the result in friendlier terms: a small add-on, called an adapter, is added to the base model. It says "a language model with billions of parameters may be LoRA fine-tuned with only several millions of parameters".

In one line: LoRA keeps the big model frozen and trains a small add-on, so tuning needs far less computing.

Try prompting first, then context, then tuning

Fine-tuning is the heaviest tool in the box, so try the lighter ones first.

The OpenAI docs on model optimization lay out a loop: write evals that measure the output, prompt the model with relevant context and instructions, and then "for some use cases, it may be desirable to fine-tune a model for a specific task". They also say of prompting: "The prompt engineering process may be all you need in order to get great results for your use case." And on the tuning page: "Only invest in fine-tuning after setting up evals."

That gives a ladder. Climb one rung at a time and stop when your evals say it is good enough.

  1. 1

    Write evals

    Make a small test set with the answers you want. Without it you cannot tell if anything helped.

  2. 2

    Improve the prompt

    Clear instructions, a few examples, a set format.

  3. 3

    Add context

    If the model lacks facts, retrieve them and put them in the prompt. That is RAG.

  4. 4

    Fine-tune

    If the behaviour is still wrong, or the prompt has grown huge, train on your own examples.

  5. 5

    Evaluate again

    Run the same evals on the tuned model. Keep it only if it clearly wins.

  • Prompting fixes

    Unclear asks and missing examples. Quick to try.

  • RAG fixes

    Missing or changing facts, read fresh each time.

  • Fine-tuning fixes

    A stubborn style, format or narrow task.

Why tune at all, if prompting can do so much? The same docs list what fine-tuning adds: you can train on more examples than fit in the context window of one request, use shorter prompts that "saves on token costs at scale and can be lower latency", and train a smaller, cheaper, faster model for one task.

A full side-by-side of RAG and fine-tuning comes later, in the Core part of this course.

What it really costs

It can look like one button, but the cost sits in the work around it.

  • Data work

    Someone has to write or collect the examples and check every one. That is people's time.

  • Training runs

    Each run uses computing power and is billed. The docs describe a loop: adjust the data, measure, repeat.

  • Evaluation

    You need evals before and after, or you cannot tell if the tuned model is better.

Then there is upkeep. A tuned model depends on its base model: the OpenAI docs say fine-tuned models "remain available for inference until their base model shuts down". Those docs also say the company is winding down its fine-tuning platform, which is closed to new users. That is one vendor at the time of writing, but it shows that offers change, so read the current docs before you plan around one.

And when your tone guide or offers change, the examples go stale. You update the data and train again. A prompt you can edit in a minute.

A made-up sum: say each example takes 5 minutes to write and check, and you want 50. That is 250 minutes, about four hours, before any training. The 5 minutes is invented, yours will differ, but the hours go into the data.

Worked example: build a training file

Let us build what you would hand to a fine-tuning job: a small dataset. Our made-up bakery wants its helper bot to always answer in one short, friendly sentence. That is a style, a fit for tuning.

The code builds six pairs into chat-style JSON lines with the json module. It checks every line: roles are system, user, assistant in that order, no message is empty, none is over 120 characters. It tries the check on a broken line. Then it shuffles with a fixed seed and splits 5 for training and 1 for validation.

No model is called and no training run happens. This is only the data work. It uses Python's standard library, so it runs free on your computer.

Python
import json
import random

SYSTEM = "You are the Daily Loaf bakery helper. Answer in one short, friendly sentence."
pairs = [
    ("When do you open?", "We open at 7am every day, see you soon!"),
    ("Do you have vegan bread?", "Yes, our seeded loaf is fully vegan, enjoy!"),
    ("Can I order a cake?", "Yes, order two days ahead and we will bake it fresh!"),
    ("Is there parking?", "Yes, there are free spots right behind the shop!"),
    ("Do you deliver?", "We deliver within 3 km, just ask at checkout!"),
    ("Is the rye loaf nut free?", "Yes, the rye loaf has no nuts, happy eating!"),
]

lines = []
for question, answer in pairs:
    example = {"messages": [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": question},
        {"role": "assistant", "content": answer},
    ]}
    lines.append(json.dumps(example))

MAX_CHARS = 120
EXPECTED_ROLES = ["system", "user", "assistant"]

def check(line):
    problems = []
    try:
        messages = json.loads(line)["messages"]
    except (ValueError, KeyError):
        return ["not valid JSON with a messages list"]
    if [m.get("role") for m in messages] != EXPECTED_ROLES:
        problems.append("roles are not system, user, assistant")
    for m in messages:
        text = m.get("content", "")
        if not text.strip():
            problems.append("empty " + str(m.get("role")))
        elif len(text) > MAX_CHARS:
            problems.append("too long: " + str(m.get("role")))
    return problems

print("Lines:")
for line in lines:
    print(line)

print()
bad = json.dumps({"messages": [{"role": "user", "content": "Hi"}, {"role": "assistant", "content": ""}]})
print("A broken line:", check(bad))

ok = [line for line in lines if not check(line)]
print("Valid lines:", len(ok), "of", len(lines))

random.Random(7).shuffle(ok)
train, validation = ok[:5], ok[5:]
print("Train:", len(train), "Validation:", len(validation))
print("Held out:", json.loads(validation[0])["messages"][1]["content"])

Output

Lines:
{"messages": [{"role": "system", "content": "You are the Daily Loaf bakery helper. Answer in one short, friendly sentence."}, {"role": "user", "content": "When do you open?"}, {"role": "assistant", "content": "We open at 7am every day, see you soon!"}]}
{"messages": [{"role": "system", "content": "You are the Daily Loaf bakery helper. Answer in one short, friendly sentence."}, {"role": "user", "content": "Do you have vegan bread?"}, {"role": "assistant", "content": "Yes, our seeded loaf is fully vegan, enjoy!"}]}
{"messages": [{"role": "system", "content": "You are the Daily Loaf bakery helper. Answer in one short, friendly sentence."}, {"role": "user", "content": "Can I order a cake?"}, {"role": "assistant", "content": "Yes, order two days ahead and we will bake it fresh!"}]}
{"messages": [{"role": "system", "content": "You are the Daily Loaf bakery helper. Answer in one short, friendly sentence."}, {"role": "user", "content": "Is there parking?"}, {"role": "assistant", "content": "Yes, there are free spots right behind the shop!"}]}
{"messages": [{"role": "system", "content": "You are the Daily Loaf bakery helper. Answer in one short, friendly sentence."}, {"role": "user", "content": "Do you deliver?"}, {"role": "assistant", "content": "We deliver within 3 km, just ask at checkout!"}]}
{"messages": [{"role": "system", "content": "You are the Daily Loaf bakery helper. Answer in one short, friendly sentence."}, {"role": "user", "content": "Is the rye loaf nut free?"}, {"role": "assistant", "content": "Yes, the rye loaf has no nuts, happy eating!"}]}

A broken line: ['roles are not system, user, assistant', 'empty assistant']
Valid lines: 6 of 6
Train: 5 Validation: 1
Held out: Can I order a cake?

Each of the six lines is one complete JSON record, as JSON Lines asks. The broken line, with no system message and an empty reply, was caught with two problems named. All 6 of 6 real lines passed. The seeded shuffle gave 5 for training and 1 for validation, and the held-out example is "Can I order a cake?". The seed is fixed, so you get the same split every run.

Real training needs at least the 10 examples the docs name, more like 50. The shape of the work is the same: write pairs, check them, hold some back.

Tip

Why hold some back? Testing on examples the model trained on is like a student who saw the exam in advance. Held-out examples tell the truth.

Quick check

Your chatbot gives wrong answers about this week's opening hours. What is the best first fix?

Recap

Prompt first, add context next, and fine-tune only when the behaviour is still wrong. Fine-tuning moves style, format and narrow tasks, not facts. When you do tune, the data is the real work.

Next: AI agents, models that plan and use tools to finish a job.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.