esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

AI engineering / Advanced / 03.28

Generating images from text

How a model turns a sentence into a picture, what you can control, where it helps in a product, and the rules to follow.

13 min readFree

From words to pictures

The last lesson, Image Understanding, ended with a promise: next, image generation, making pictures from words. Here it is. Until now the model looked at a picture and told you about it. Now it works the other way round. You write a sentence, and a picture comes back.

Think of a very fast illustrator who never sleeps. You hand over a short note, and a minute later a drawing appears. The note matters a lot. So do the rules about what you may ask for.

By the end you will know how these models work in one plain paragraph, what you can control, how two real image APIs look, where images help a product, and what to watch out for. You will also run two small programs, with no model and no cost.

Key words

Image generation
Making a new picture from a written description.
Prompt
The text you write to say what the picture should show.
Diffusion model
A kind of model that makes a picture by starting from random speckles and cleaning them up step by step.
Noise
Random speckles, like the snow on an old TV with no signal.
Mask
A second picture that marks which part of a photo may change and which part must stay.
Base64
A way to write picture bytes as ordinary text, so they can travel inside a web reply.
Provenance
The history of a file: who made it and what was done to it.

How a picture grows out of noise

Here is the whole idea in one paragraph. Wikipedia describes a diffusion model as having two major components: the forward diffusion process and the reverse sampling process. During training, real pictures are covered in more and more noise. The model learns to undo that damage. Wikipedia says these models typically involve "training a neural network to sequentially denoise images corrupted by Gaussian noise". After training, you start from pure noise and let the model clean it up in many small steps until a picture is left. The text you write steers the cleaning. Wikipedia notes that image generators of this kind "typically combine diffusion models with other models, such as text-encoders and cross-attention modules to allow text-conditioned generation".

A daily-life picture helps. Imagine a window covered in thick fog. You wipe it, and a little more of the street shows each time. A diffusion model wipes the fog away one small step at a time, and your prompt tells it which street to reveal.

  1. 1

    Write the prompt

    Your words are turned into numbers, the way any language model does it. The Tokenization lesson in Machine Learning Foundations goes deeper.

  2. 2

    Start from noise

    The model begins with random speckles. Nothing about the final picture exists yet.

  3. 3

    Clean up, many times

    Each step removes a little noise, nudged by the numbers from your prompt.

  4. 4

    Get the picture

    After the last step, the speckles have become an image.

Because it starts from random noise, the same prompt can give a different picture each time. That is normal, and it is also why you cannot promise a customer one exact result.

Why it works in a smaller space

Cleaning every pixel of a big picture is slow. The arXiv paper on latent diffusion points out that models working directly on pixels often need "hundreds of GPU days" to train, and that running them is expensive. A GPU is the kind of chip that does this heavy maths.

The fix in the paper is to apply the diffusion "in the latent space of powerful pretrained autoencoders". A latent space is a compact, squeezed version of a picture, like a thumbnail that still holds the important facts. The model cleans the small version, and a decoder blows it back up. The abstract adds that adding cross-attention layers turns these models into generators for inputs "such as text or bounding boxes", and that the approach "significantly" reduces computing needs compared with pixel-based models.

What you can control

You do not get a dial for every detail. You get a short list of settings, and most of the quality comes from the prompt. These are the controls the two official guides in the sources list.

  • The prompt

    Your description. A clear subject, a style and a mood beat a vague wish. This is your main tool.

  • Size and shape

    The OpenAI guide lists recommended sizes of 1024x1024 (square), 1536x1024 (landscape) and 1024x1536 (portrait). The Gemini example sets an aspect ratio of 1:1 and an image size of 1K.

  • Quantity and quality

    OpenAI's n setting asks for several images in one request, and by default the API returns a single image. Its quality setting runs from low to higher levels, and the guide suggests low for quick drafts.

There are more. The OpenAI guide also lists the output format, a compression level for JPEG and WebP, and a background that can be transparent, opaque or automatic. For custom sizes it says width and height must be multiples of 16.

Editing is a second skill, separate from making a new picture. The OpenAI guide describes an edits endpoint that modifies existing images with a new prompt, "either partially or entirely". For a partial edit you pass a mask. The guide says the image and the mask must be the same format and size, under 50MB, and the mask must contain an alpha channel. Think of a mask as masking tape on a wall: you paint only where the tape is not.

Gemini works in a chat style. Its page says you can provide an image and use text prompts to add, remove or modify elements, change the style, or adjust the colour grading. It also describes mixing several reference images, with up to 14 images in total for one of its models. Reference images are example pictures you hand over to show the look you want.

Tip

Do not trust a count. The Gemini page warns that the model won't always follow the exact number of image outputs you ask for. Check how many came back.

Two real APIs, checked

The course plan lists these two topics as the DALL-E API and the NanoBanana API. Here is what the official pages we fetched actually say, with no extras.

OpenAI first. Its current Image generation guide says the API lets you generate and edit images from text prompts, and it names two models, gpt-image-2.5-sunburst and gpt-image-2.5-flare. It describes an Image API with two endpoints, Generations and Edits, and a Responses API that can generate images inside a conversation. The guide does not use the name DALL-E. Wikipedia does: it names DALL-E and Stable Diffusion as diffusion-based image generators with "widespread commercial interest".

Below is the guide's Python example, copied as shown. The image does not come back as a file. It comes back as base64 text, which the code decodes into bytes and saves. It needs an account and a key, so we do not run it here.

Text
# From the OpenAI "Image generation" guide, shown without its output (not run here)
from openai import OpenAI
import base64

client = OpenAI()

prompt = """
A children's book drawing of a veterinarian using a stethoscope to
listen to the heartbeat of a baby otter.
"""

result = client.images.generate(model="gpt-image-2.5-sunburst", prompt=prompt)

image_base64 = result.data[0].b64_json
image_bytes = base64.b64decode(image_base64)

# Save the image to a file
with open("otter.png", "wb") as f:
    f.write(image_bytes)

Now Google. Its Gemini API page says "Nano Banana is the name for Gemini's native image generation capabilities", and it lists several Nano Banana models. It also says all generated images include a SynthID watermark. The shape is the same as before: a model name, a prompt, and base64 data to decode.

Text
# From the Gemini API "Nano Banana image generation" page, shown without its output (not run here)
from google import genai
from PIL import Image
import base64

client = genai.Client()

interaction = client.interactions.create(
    model="gemini-nano-banana-2.1",
    input="Create a picture of a nano banana dish in a fancy restaurant with a Gemini theme",
)

with open("generated_image.png", "wb") as f:
    f.write(base64.b64decode(interaction.output_image.data))

Tip

These APIs were checked on 16 November 2026. Model names and settings change often, so open the official page before you build.

Where images help a product

Image generation does not replace a designer. It speeds up the first draft and covers the pictures nobody has time to draw.

  • Mock-ups

    Fill a screen design with believable pictures before the real photos exist, so a client can judge the layout.

  • Illustrations

    Spot drawings for an article, a slide or an empty state, in one style you name in the prompt.

  • Variations

    Ask for several takes of one idea, pick the best, and edit it with a mask.

Every real model call is slower than a normal web request, costs money each time, can give a different picture on each run, and can fail or refuse. So a good product saves what it likes, shows a loading state, and has a plan for the day a request is blocked.

Risks and rules

Pictures carry more risk than text, because people trust what they see. Here are the points to keep in mind.

Rights. The Gemini page reminds you to make sure you have the necessary rights to any images you upload, and not to generate content that infringes on others' rights, including images that deceive, harass or harm. Whether a generated image breaks copyright is a legal question that this lesson does not answer. Ask a lawyer before you ship something that matters.

People's likeness. As plain good practice, never ask for a picture of a real person without their permission. A convincing fake of someone can hurt them, and it can get your product removed.

Provider rules. OpenAI says all prompts and generated images are filtered in line with its content policy, and its API can return a moderation_blocked error. The guide says not to retry such user errors automatically without changing the prompt or input images. Your product should show a calm message and let the person try a different request. Provider rules can differ, so read each one.

Labels. Be honest when a picture is made by AI. Two tools help. Gemini adds a SynthID watermark to every generated image. C2PA is an open standard for the origin and edits of digital content, called Content Credentials, which it says function "like a nutrition label for digital content". A label helps, but no single tool is a full answer, so do not rely on one alone.

Quick check

A request to your image API is blocked by the provider's content filter. What is the best next step?

Try it: a safe prompt builder

Many image bugs start with a messy prompt. A small builder fixes that. It joins fields into one clean line, checks the size against a list of allowed sizes, and rejects prompts with blocked words. Notice it calls no model at all. It is plain string work, which is why it runs for free.

  1. 1

    Collect fields

    Subject, style, lighting and size, each in its own box.

  2. 2

    Validate

    Reject a size that is not on the list, and any blocked term.

  3. 3

    Build

    Join the fields into one prompt, ready to send.

Python
ALLOWED_SIZES = {"1024x1024": "square", "1536x1024": "wide", "1024x1536": "tall"}
BLOCKED_TERMS = ["real celebrity", "logo of", "in the style of living artist"]


def build_prompt(subject, style, lighting, size):
    if size not in ALLOWED_SIZES:
        raise ValueError("size " + size + " is not allowed")
    text = (subject + ", " + style + ", " + lighting + " lighting, "
            + ALLOWED_SIZES[size] + " frame")
    for term in BLOCKED_TERMS:
        if term in text.lower():
            raise ValueError("blocked term: " + term)
    return text


requests = [
    ("a red teapot on a desk", "flat illustration", "soft morning", "1024x1024"),
    ("a phone app home screen on a table", "clean mock-up", "bright studio", "1024x1536"),
    ("a quiet harbour at dawn", "watercolour", "golden hour", "1536x1024"),
    ("a real celebrity at a cafe", "photo", "natural", "1024x1024"),
]
for fields in requests:
    try:
        print("OK  ", build_prompt(*fields))
    except ValueError as err:
        print("NO  ", err)

Output

OK   a red teapot on a desk, flat illustration, soft morning lighting, square frame
OK   a phone app home screen on a table, clean mock-up, bright studio lighting, tall frame
OK   a quiet harbour at dawn, watercolour, golden hour lighting, wide frame
NO   blocked term: real celebrity

Three prompts passed. The fourth was rejected before any money was spent, because it asked for a real celebrity. In a real product the blocked list would be longer and it would only be one layer of checking. The three sizes here are the same ones the OpenAI guide recommends.

Images come back as data

Your product never receives a drawing. It receives bytes, which are numbers, and your code saves them as a file. A PNG is just a header, some compressed rows of colours and a closing mark. This code builds a tiny one by hand with the standard library.

Read this carefully: fake_model is a stand-in. It ignores the prompt and paints a colour gradient, so the code runs free and offline. No model is called. A real model is slower, costs money for every request, can give a different answer each run, and can fail or refuse.

Python
import struct
import zlib


def chunk(kind, data):
    body = kind + data
    return struct.pack(">I", len(data)) + body + struct.pack(">I", zlib.crc32(body))


def make_png(width, height, pixels):
    rows = b""
    for y in range(height):
        rows += b"\x00" + bytes(pixels[y * width * 3:(y + 1) * width * 3])
    header = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0)
    return (b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", header)
            + chunk(b"IDAT", zlib.compress(rows)) + chunk(b"IEND", b""))


def fake_model(prompt):
    # Stand-in: ignores the prompt and paints a colour gradient.
    pixels = []
    for y in range(8):
        for x in range(8):
            pixels += [30 + x * 28, 90, 220 - x * 25]
    return make_png(8, 8, pixels)


data = fake_model("a red teapot on a desk")
print(len(data), "bytes")
print(data[:8])
print(data[1:4].decode())

Output

94 bytes
b'\x89PNG\r\n\x1a\n'
PNG

The result is 94 bytes. The first eight bytes are the standard PNG signature, and the word PNG sits just after the first byte. A real picture is much bigger, but the idea is the same: bytes in, file out.

Quick check

In the OpenAI example, what does the code do with b64_json?

What to remember

A diffusion model makes a picture by cleaning noise in small steps, steered by your words. You control the prompt, the size, how many images, the quality and edits with a mask. Images come back as data you save. Check provider rules, respect people and rights, and label AI pictures.

Next: speech-to-text, turning recordings into words.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.