esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Machine Learning Foundations / Advanced / 03.25

Keras: the friendly way to stack layers

Build a neural network in three steps: build, compile, fit. See the real Keras lines, then watch a tiny stack of layers run in plain Python.

14 min readFree

What Keras is

Last lesson you met TensorFlow, a big engine for training neural networks. An engine is powerful, but nobody wants to build a car from loose bolts every morning. Keras is the dashboard and steering wheel. You say what you want in a few readable lines, and the engine underneath does the heavy work.

The Keras docs describe it in one sentence: 'Keras is a deep learning API written in Python and capable of running on top of either JAX, TensorFlow, or PyTorch.' An API is a set of buttons that code can press. Deep learning is the kind of machine learning that uses neural networks with many layers, which you met in the earlier lessons.

Wikipedia adds a little history. Keras is an open-source library that provides a Python interface for artificial neural networks. Its primary author is François Chollet, and it was first released on 27 March 2015. The name comes from the Ancient Greek word for 'horn'. It was 'first independent software, then integrated into the TensorFlow library, and later added support for more.'

Why is it called friendly? The About page lists three promises. Keras is 'Simple - but not simplistic', since it 'reduces developer cognitive load', which means it keeps your head free for the real problem. It is 'Flexible', because simple jobs stay quick while harder jobs remain possible. And it is 'Powerful': the page says it is used by organizations including NASA, YouTube and Waymo. The Keras 3 page says its workflows run 'from the very high-level to the very low-level', so you can start easy and go deeper later.

Key words

Layer
One step of a network. It takes numbers in, changes them and passes new numbers out. Like one stage on a factory belt.
Model
All the layers together, plus the rules for learning. It is the thing you train and then use.
Backend
The engine under Keras that does the real number crunching, such as TensorFlow, JAX or PyTorch.
Optimizer
The part that nudges the weights after each mistake, so the next guess is a bit better.
Loss
A single score for how wrong the model is. Training tries to push it down.
Epoch
One full pass through all your training data.
Batch
A small handful of examples used for one weight update, instead of the whole pile at once.

Keras 3 and its backends

The current version is called Keras 3. The announcement page says it 'is a full rewrite of Keras that enables you to run your Keras workflows on top of either JAX, TensorFlow, PyTorch, or OpenVINO (for inference-only)'. Inference means using a finished model to make predictions, with no training. The Getting Started page lists the backends you can pick for Keras code as 'jax', 'torch' and 'tensorflow'.

  1. 1

    Install Keras

    The Getting Started page gives one line: pip install --upgrade keras. Run it in a terminal, ideally inside a virtual environment, as in the scikit-learn lesson.

  2. 2

    Install one backend

    The page says to use Keras 3 you will also need to install a backend framework, such as JAX, PyTorch or MLX, with links to each install guide. TensorFlow is listed there too.

  3. 3

    Choose it with KERAS_BACKEND

    Set the environment variable KERAS_BACKEND to "jax", "torch" or "tensorflow", or edit the config file at ~/.keras/keras.json.

  4. 4

    Do it before you import

    The docs are firm: 'The backend must be configured before importing Keras, and the backend cannot be changed after the package has been imported.'

The three steps every Keras model follows

Every Keras project, from a toy to a giant, follows the same short path. Picture baking bread. First you shape the dough, then you set the oven, then you bake and taste.

Build: stack the layers. Compile: choose how learning works. Fit: train on data. After that you evaluate (check the score on data the model has never seen) and predict (use it on new data). Here is the whole path as a picture.

Text
build  ->  compile  ->  fit  ->  evaluate  ->  predict
layers     optimizer    epochs    score on     answers for
           loss         batches   new data     brand-new rows
           metrics

Quick check

In what order do you do the first three Keras steps?

Step 1: build with Sequential and Dense

The core data structures of Keras are layers and models. The simplest type of model is the Sequential model, which the About page calls 'a linear stack of layers'. The Sequential guide says it 'is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor'. A tensor is a box of numbers, which you met in the lesson on scalars, vectors and tensors.

The first snippet is copied from the Sequential guide. It builds three layers. It is shown as the docs give it and was not run here.

Text
# From the Keras Sequential model guide, shown without its output (not run here)
model = keras.Sequential(
    [
        layers.Dense(2, activation="relu"),
        layers.Dense(3, activation="relu"),
        layers.Dense(4),
    ]
)

Read it line by line. keras.Sequential(...) makes an empty stack and fills it with the list you give it. Each layers.Dense(...) is a layer where every output looks at every input. The number is how many outputs the layer makes: 2, then 3, then 4. activation="relu" is a small rule applied to each output, which keeps positive numbers and turns negative ones into zero. The last layer has no activation, so its numbers pass straight out.

A Dense layer does simple maths. Each output is the inputs times their weights, added up, plus a bias, passed through the activation. In symbols:

output=f(∑ixi wi+b)\text{output} = f\left(\sum_i x_i\,w_i + b\right)

In words: multiply each input x by its weight w, add them all, add the bias b, then run the result through the activation f. The Sequential guide shows the shapes. A Dense(3) layer fed four numbers gets weights of shape (4, 3) and a bias of shape (3,), which is one weight for every input-and-output pair, and one bias per output.

Tip

Keras layers do not make their weights until they know the input size. The guide says that when you create a Sequential model without an input shape, it is not 'built' and has no weights until it first sees some input data. Start with keras.Input(...) if you want to see the model's summary right away.

You can also build the same stack one layer at a time. The guide says it is 'useful to incrementally stack layers with add() and frequently print model summaries', and that a Sequential model 'behaves very much like a list of layers'.

Step 2: compile with optimizer, loss and metrics

Compile does not run anything yet. It is like setting the oven before you put the bread in. The intro guide says: 'We use the compile() method to specify the optimizer, loss function, and the metrics to monitor.'

  • Optimizer

    The method that updates the weights after each batch. 'sgd' is the plain one, short for stochastic gradient descent. You can also pass an object and set its learning rate, which is how big each nudge is.

  • Loss

    The score of wrongness that training pushes down. The About page uses 'categorical_crossentropy'. The Loss functions lesson explains why cross-entropy suits categories and why mean squared error suits numbers.

  • Metrics

    Scores for you to read, such as 'accuracy'. They do not steer training. The loss does that. You watch metrics the way you watch a speedometer.

The second snippet comes from the About Keras 3 page. It shows compile, fit, evaluate and predict together. Again, it is the docs' text and was not run here.

Text
# From the About Keras 3 page, shown without its output (not run here)
model.compile(
    loss='categorical_crossentropy',
    optimizer='sgd',
    metrics=['accuracy'],
)

# x_train and y_train are Numpy arrays
model.fit(x_train, y_train, epochs=5, batch_size=32)

loss_and_metrics = model.evaluate(x_test, y_test, batch_size=128)
classes = model.predict(x_test, batch_size=128)

The compile call picks 'categorical_crossentropy' as the loss, 'sgd' as the optimizer and 'accuracy' as the metric. The docs add that 'if you need to, you can further configure your optimizer', so the short names are a shortcut and nothing more. They also say Keras keeps simple things simple while letting you take full control when you need to.

Step 3: fit, then evaluate and predict

Fit is where learning happens. You hand over the questions (x_train) and the right answers (y_train). The About page says fit lets you 'iterate on your training data in batches'.

  • epochs=5

    Go through the whole training set five times. One trip through all the data is one epoch, like reading a textbook once.

  • batch_size=32

    Learn from 32 examples at a time, then update the weights. Small batches mean many small updates. The Back propagation lesson showed one such update by hand.

  • validation_split

    Hold back a slice of the training data and never learn from it. After each epoch, Keras scores the model on that slice, so you can see whether it is truly learning or just memorising.

That third item is the same idea as the train and test data lesson, applied inside fit. The intro guide says: 'We'll set aside a validation split of 15% of the data during training to monitor generalization on unseen data.' Generalization means doing well on data the model has not studied.

Here is some plain Python that does the arithmetic Keras does for you. With 1,000 rows, a batch size of 128 and a 15% validation split, how many updates happen? It was run with python3 -I.

Python
import math

n = 1000
batch_size = 128
validation_split = 0.15
epochs = 3

n_val = int(n * validation_split)
n_train = n - n_val
steps = math.ceil(n_train / batch_size)

print("training rows:", n_train, "| validation rows:", n_val)
print("batches per epoch:", steps)
print("weight updates in", epochs, "epochs:", steps * epochs)
print("rows in the last batch:", n_train - (steps - 1) * batch_size)

Output

training rows: 850 | validation rows: 150
batches per epoch: 7
weight updates in 3 epochs: 21
rows in the last batch: 82

Of the 1,000 rows, 150 are held back for checking and 850 are used for learning. Six full batches of 128 use 768 rows, and a seventh, smaller batch takes the last 82. Each epoch is therefore seven weight updates. After fit, evaluate gives the loss and metrics on test data the model never saw, and predict returns the model's answers for new rows.

Quick check

What does validation_split=0.15 do in fit?

A tiny Sequential in plain Python

You cannot run Keras here, but you can see the idea behind it. The next code is a stand-in written with plain Python functions. It has a Dense maker, ReLU and sigmoid, and a Sequential that passes each row through the layers in order. The weights are fixed numbers I chose, so nothing is learned. This is only the forward pass, the part where numbers flow through the stack.

Notice how the shape mirrors the docs snippet: Sequential takes a list of Dense layers, and each Dense has a size and an activation. Sigmoid squashes any number into the range between 0 and 1, so the answer can be read as a probability-like score. It was run with python3 -I.

Python
import math


def relu(x):
    return max(0.0, x)


def sigmoid(x):
    return 1 / (1 + math.exp(-x))


def Dense(weights, biases, activation=None):
    """One layer: every output = activation(sum(input * weight) + bias)."""
    def layer(row):
        out = []
        for j in range(len(biases)):
            total = sum(row[i] * weights[i][j] for i in range(len(row))) + biases[j]
            out.append(activation(total) if activation else total)
        return out
    return layer


def Sequential(layers):
    """A plain stack: feed each row through the layers, one after another."""
    def model(rows):
        results = []
        for row in rows:
            for layer in layers:
                row = layer(row)
            results.append(row)
        return results
    return model


model = Sequential([
    Dense([[0.5, -1.0], [0.25, 0.5], [-0.5, 1.0]], [0.0, 0.5], relu),
    Dense([[1.0], [-0.5]], [0.1], sigmoid),
])

x = [[1.0, 1.0, 1.0],
     [2.0, 0.0, 1.0],
     [3.0, -1.0, 0.0]]
y = model(x)
for row, out in zip(x, y):
    print(row, "->", round(out[0], 3))
print("samples:", len(y), "| outputs each:", len(y[0]))

Output

[1.0, 1.0, 1.0] -> 0.463
[2.0, 0.0, 1.0] -> 0.646
[3.0, -1.0, 0.0] -> 0.794
samples: 3 | outputs each: 1

Walk through the first row, [1, 1, 1]. The first layer has two outputs. The first output is 1 times 0.5, plus 1 times 0.25, plus 1 times negative 0.5, plus a bias of 0, which makes 0.25. ReLU keeps it. The second output is negative 1 plus 0.5 plus 1, plus a bias of 0.5, which makes 1.0. The second layer then computes 0.25 times 1.0, plus 1.0 times negative 0.5, plus 0.1, which is negative 0.15. Sigmoid turns that into 0.463.

When a plain stack is not enough

Sequential has limits, and the guide lists them. It is not appropriate when your model has multiple inputs or multiple outputs, when a layer has multiple inputs or outputs, when you need layer sharing, or when you want a non-linear topology, such as a residual connection or a multi-branch model. In plain words: a Sequential model is a single road, and some networks need forks and merges.

For those, use the Functional API. The About page says it 'allows to build arbitrary graphs of layers', and the intro guide marks it as '(most typical)' among the three ways to build. You treat layers like functions: you start from an Input, pass it through a layer to get a new value, and pass that on, wiring the layers in any shape you like. A model is then drawn from where the input starts to where the outputs end. The Sequential guide adds that once a Sequential model is built, it behaves like a Functional API model.

  • Sequential

    A single road. Easiest to read. Use it when each layer feeds only the next one.

  • Functional API

    A map of roads. Use it when the layers branch, join or share parts. The docs call it the most typical.

  • Subclassing

    Writing your own model from scratch. The docs reserve it for advanced use cases.

Callbacks: helpers that watch training

A callback is a helper you hand to fit. It gets called at set moments, such as the end of each epoch. The third snippet comes from the intro guide. ModelCheckpoint saves the model after each epoch, and EarlyStopping stops training when the validation loss stops getting better. patience=2 means wait two epochs for an improvement before giving up. It is shown as the docs give it and was not run here.

Text
# From the Keras Introduction for engineers, shown without its output (not run here)
batch_size = 128
epochs = 20

callbacks = [
    keras.callbacks.ModelCheckpoint(filepath="model_at_epoch_{epoch}.keras"),
    keras.callbacks.EarlyStopping(monitor="val_loss", patience=2),
]

model.fit(
    x_train,
    y_train,
    batch_size=batch_size,
    epochs=epochs,
    validation_split=0.15,
    callbacks=callbacks,
)

Here is the early-stopping idea in plain Python. The made-up validation losses fall for three epochs, then wobble. The loop counts epochs without a new best score and stops when the count reaches the patience. It was run with python3 -I.

Python
best, wait, patience = float("inf"), 0, 2
val_loss = [0.52, 0.31, 0.22, 0.23, 0.24, 0.20]
for epoch, v in enumerate(val_loss, start=1):
    if v < best:
        best, wait = v, 0
    else:
        wait += 1
    print("epoch", epoch, "val_loss", v, "waited", wait)
    if wait >= patience:
        print("stop: no improvement for", patience, "epochs; best was", best)
        break

Output

epoch 1 val_loss 0.52 waited 0
epoch 2 val_loss 0.31 waited 0
epoch 3 val_loss 0.22 waited 0
epoch 4 val_loss 0.23 waited 1
epoch 5 val_loss 0.24 waited 2
stop: no improvement for 2 epochs; best was 0.22

Quick check

A Keras model has two separate inputs. Which way of building it fits the docs' advice?

What to remember

Pick one backend, set KERAS_BACKEND before you import Keras, and keep the same code. Build, compile, fit: say what each does in your own words and you have this lesson.

Next: PyTorch, building networks in a plain Python style.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.