esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

AI engineering / Beginner / 01.11

Structured Output

How to ask a model for data your program can use, not just a paragraph, and how to check every reply before your product trusts it.

13 min readFree

Answers your code can read

The last lesson, Chain-of-thought prompting, ended with a promise: next, structured output, getting answers your code can read. Here it is.

Think about asking a friend for a restaurant tip. They say, "Oh, you'd love the little place near the station, the noodles are great, a bit loud though." You can use that. Now imagine a program has to use that sentence. Where is the name? Where is the rating? A person can read between the lines. A program cannot.

This lesson is about asking a model for data your program can use, not just a nice paragraph. By the end you will know what JSON is, three ways to ask for it, why you must always check what comes back, and the three mistakes models make most. You will run three short Python programs. None of them calls a real model.

Key words

Structured output
A model reply in a fixed shape with named fields, such as JSON, instead of free-flowing prose.
JSON
A plain-text way to write data: named fields and their values, in a layout both people and programs can read.
Field
One named piece of data, like rating or product.
Schema
A written description of the shape you expect: which fields, what type each one is, and which values are allowed.
Validate
To check that data matches the shape you expected before your program uses it.
Parse
To read text and turn it into data your program can work with.

Why a product needs fields, not prose

Say you are building a page that shows product reviews. For each review you want a star rating, a mood label, and a one-line summary. You send each review to a model and ask it to analyse it.

If the model answers in prose, you get a different sentence every time. "This one seems pretty happy, maybe four stars." Your page cannot put that into a star widget. You would have to guess where the number is.

If the model answers with fields, your page can read rating and draw four stars. A designer can then decide how a positive review looks and how a negative one looks, because the mood is always one of a few known words.

Tip

Ask yourself what your screen or your database needs. List the fields first. Then ask the model for exactly those fields, and nothing else.

JSON and schemas in plain words

JSON stands for JavaScript Object Notation. Its own home page, json.org, calls it "a lightweight data-interchange format" that is "easy for humans to read and write" and "easy for machines to parse and generate". Many programming languages can read it.

The same page says JSON is built on two structures: "a collection of name/value pairs" and "an ordered list of values". The first is called an object. It looks like a small form with labelled boxes. The second is called an array, which is just a list.

The types you will meet

The JSON Schema site lists the basic pieces: object, array, number, string, boolean and null. In everyday words:

  • Text and numbers

    A string is text in double quotes, like "Desk lamp". A number has no quotes, like 4. Mixing them up is a common bug: "4" with quotes is text, not a number.

  • Yes or no, and nothing

    A boolean is true or false. Null means there is no value.

  • Boxes and lists

    An object holds named fields inside curly braces. An array holds an ordered list inside square brackets.

Here is a tiny review as JSON: {"rating": 4, "sentiment": "positive"}. It is an object with two fields. The rating is a number. The sentiment is text.

What a schema adds

JSON can hold almost any shape. The JSON Schema site says that when an application asks for a record, "we need to know what fields are expected, and how the values are represented". That is exactly the job of a schema.

A schema is the form you hand out before anyone fills it in. It says: these fields, these types, and these allowed values. You can then check any reply against it. JSON Schema is a standard way to write that description. In the worked example below we write a small version in plain Python so you can see how the check works.

Three ways to ask, weakest to strongest

There are three common ways to get structured output from a model. Each one is stronger than the one before, and you can combine them.

  1. 1

    Ask in the prompt

    Just say it: "Reply with JSON only, with the fields rating and sentiment." This is free and quick. It is also the weakest. The model may add friendly words around the JSON, or drift from your field names. The Prompt Engineering lesson covers the habits that help, like being clear and saying what format you want.

  2. 2

    Give an example

    Show one finished reply in your prompt: "Here is a good answer: {"rating": 4, "sentiment": "positive"}". The hope is that the model copies the shape. It often helps, and it is still only a hope, with no guarantee.

  3. 3

    Use the provider's JSON mode or schema feature

    Some model providers build the shape into the service. You pass a schema, and the service holds the model to it. The next section describes what one provider's documentation says. Other providers have their own versions, so read your provider's current page.

Tip

These ways stack. A clear prompt, plus an example, plus a provider feature if you have one, plus your own check at the end. Each layer catches what the one before missed.

What a provider feature does, and does not do

OpenAI's documentation page on Structured Outputs describes two features. Other providers offer similar ones, so treat this as one example. This page is the source for everything in this section.

The first is JSON mode. The page says it "is a more basic version of the Structured Outputs feature". It makes the output valid JSON, "except for in some edge cases that you should detect and handle appropriately". It does not promise your fields. The page also says that with JSON mode you must still tell the model, in a message, to produce JSON.

The second is Structured Outputs. The page says it "constrains completed, non-refusal model output to your supplied JSON Schema". You hand over a schema, and the service holds the reply to it. The page lists benefits: required fields and enum values follow your schema, refusals can be detected by your code, and you need no "strongly worded prompts to achieve consistent formatting". An enum is a short fixed list of allowed values, like positive, neutral or negative.

Read the fine print, though. The same page says: "Your application still needs to handle refusals, incomplete responses, and API errors, and validate any business rules that the schema does not express." A schema can say rating is a number. It cannot know that a review of a lamp should not mention a different product.

A real model call also costs money for every token, is slower than a function in your own code, and can fail. Even with a schema feature, plan for the day it does.

Always validate

Whatever way you ask, treat the reply as untrusted until your own code has checked it. Think of a bouncer at a door with a guest list. The reply may be well dressed, but the bouncer still checks the list.

Validation has three small steps, and you will see all three in the code below.

  1. 1

    Parse

    Turn the text into data. In Python, json.loads does it. The Python documentation says that if the text is not a valid JSON document, a JSONDecodeError will be raised. So catch that error.

  2. 2

    Check the shape

    Are all required fields there? Is each one the right type? Is each value one of the allowed ones?

  3. 3

    Decide what to do on failure

    Retry, repair, or fall back. A retry means asking the model again. A fallback means using a safe default, or showing a message instead of a wrong answer.

First, the parse step by itself. We try a clean reply and a chatty one:

Python
import json

good = '{"rating": 4, "sentiment": "positive"}'
data = json.loads(good)
print(data["rating"], type(data["rating"]).__name__)

chatty = 'Sure! Here is the JSON: {"rating": 4, "sentiment": "positive"}'
try:
    json.loads(chatty)
except json.JSONDecodeError as error:
    print("failed:", error)

Output

4 int
failed: Expecting value: line 1 column 1 (char 0)

The clean reply becomes data, and the rating comes out as a whole number, int. The chatty reply fails, because the text starts with the word Sure, not with a curly brace. The error message says it expected a value at the very start. Remember this one: it is the most common failure.

Three failures you will see

  • Extra text around the JSON

    The model says "Sure! Here you go:" and then the JSON, then "Hope that helps!". A person is fine with that. json.loads is not. Fix it by finding the {...} block inside the text.

  • Missing fields

    You asked for four fields and got three. If your page needs the mood label and it is not there, you must notice before the page breaks.

  • Wrong types or values

    The rating arrives as "four" instead of 4, or the mood is "super happy" when only three words are allowed. The JSON is valid, and your page still cannot use it.

Tip

Valid JSON is not the same as correct JSON. Parsing only proves the text is well formed. Checking the fields proves it is usable.

Worked example: a review checker

Now we build the checker. The schema has four fields: product (text), rating (a whole number from 1 to 5), sentiment (one of three words) and summary (text). The program defines the schema, a validate function that returns a list of problems, and an extract_json function that finds the first {...} block in messy text.

Important: the three replies below are stand-ins. We wrote them by hand to look like what a model might send back. No real model is called, so this code runs free and gives the same result every time. A real model is slower, costs money for every token, can give a different answer each run, and can fail or refuse. We chose one good reply, one with chatty text around it, and one with a field missing.

Python
import json

# The schema, written as a plain dictionary: field name -> (type, allowed values or None)
SCHEMA = {
    "product": (str, None),
    "rating": (int, [1, 2, 3, 4, 5]),
    "sentiment": (str, ["positive", "neutral", "negative"]),
    "summary": (str, None),
}


def validate(data):
    """Return a list of problems. An empty list means the reply is fine."""
    if not isinstance(data, dict):
        return ["the reply is not a JSON object"]
    problems = []
    for name, (kind, allowed) in SCHEMA.items():
        if name not in data:
            problems.append(f"missing field: {name}")
        elif not isinstance(data[name], kind) or isinstance(data[name], bool):
            problems.append(f"{name} should be {kind.__name__}")
        elif allowed is not None and data[name] not in allowed:
            problems.append(f"{name} must be one of {allowed}")
    return problems


def extract_json(text):
    """Return the first {...} block in the text, or None if there is none."""
    start = text.find("{")
    if start == -1:
        return None
    try:
        data, _ = json.JSONDecoder().raw_decode(text[start:])
    except json.JSONDecodeError:
        return None
    return data


# Stand-in replies, written by hand. No real model is called here.
REPLIES = {
    "good": '{"product": "Desk lamp", "rating": 5, "sentiment": "positive", "summary": "Bright and sturdy."}',
    "chatty": 'Sure! Here is your JSON:\n{"product": "Desk lamp", "rating": 4, "sentiment": "positive", "summary": "Bright, a bit pricey."}\nHope that helps!',
    "missing": '{"product": "Desk lamp", "rating": 2, "summary": "Flickers after a week."}',
}

for label, reply in REPLIES.items():
    print(f"--- {label} reply")
    try:
        data = json.loads(reply)
    except json.JSONDecodeError:
        print("not clean JSON, trying to find the {...} block")
        data = extract_json(reply)
        if data is None:
            print("gave up: no JSON found")
            continue
    problems = validate(data)
    print("ok:" if not problems else "rejected:", problems or data)

Output

--- good reply
ok: {'product': 'Desk lamp', 'rating': 5, 'sentiment': 'positive', 'summary': 'Bright and sturdy.'}
--- chatty reply
not clean JSON, trying to find the {...} block
ok: {'product': 'Desk lamp', 'rating': 4, 'sentiment': 'positive', 'summary': 'Bright, a bit pricey.'}
--- missing reply
rejected: ['missing field: sentiment']

Read the output reply by reply. The good reply parses and passes. The chatty reply fails a plain parse, so the program looks for the {...} block, finds it, and then it passes the check. The third reply is valid JSON, but the check reports "missing field: sentiment". The program reports that error instead of showing the page a half-empty review.

Two small details in the code are worth knowing. The extract step uses raw_decode from Python's json module. The Python documentation says it decodes "a JSON document from s (a str beginning with a JSON document)" and can be used for "a string that may have extraneous data at the end". That is why the chatty reply works: text after the closing brace is ignored. Also, the type check rejects true and false when a number is expected, because in Python a boolean counts as a number unless you say otherwise.

Retry, then fall back

When validation fails, a program has options. Repair what it can, like stripping the text around the JSON. Ask the model again, sometimes with a note saying what was wrong. Or stop, and show something safe.

Here is a retry loop. The fake_model function is a stand-in, so the code runs free. It fails on the first call and succeeds on the second, which is not how a real model behaves. Real models can fail in other ways, give a different answer each run, refuse, or cost money for every call. That is why the loop stops after three tries instead of running forever.

Python
import json

calls = 0


def fake_model(messages):
    """A stand-in. It fails the first time and gives clean JSON the second time."""
    global calls
    calls += 1
    if calls == 1:
        return "Great question! The rating is 4."
    return '{"rating": 4}'


messages = [{"role": "user", "content": "Rate this review. Reply with JSON only."}]
for attempt in range(1, 4):
    reply = fake_model(messages)
    try:
        data = json.loads(reply)
        print(f"attempt {attempt}: got {data}")
        break
    except json.JSONDecodeError:
        print(f"attempt {attempt}: not JSON, asking again")
        messages.append({"role": "user", "content": "That was not valid JSON. Reply with JSON only."})
else:
    print("giving up: use a safe default")

Output

attempt 1: not JSON, asking again
attempt 2: got {'rating': 4}

The first attempt returns chatty text, so the program notes it and asks again. The second attempt returns clean JSON, so the loop stops. If all three attempts had failed, the else branch would run, and you would use a safe default.

Tip

Always cap the number of retries. Each retry of a real model costs tokens and time. Two or three tries is plenty for most apps.

Check yourself

Quick check

A reply is valid JSON, but the rating is the text "four" and your page expects a number. What now?

Quick check

Which way of asking for structured output has the weakest guarantee?

What to take with you

Structured output turns a model's words into data. Ask for fields, give an example, and use a provider's schema feature if you have one. Then check every reply yourself, because even the strongest feature leaves some checks to your code.

The Tokenization lesson in Machine Learning Foundations goes deeper on why every extra word of instructions or schema costs tokens. Keep your schemas small.

Next: function calling, letting a model ask your code to do things.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.