esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Machine Learning Foundations / Advanced / 03.30

Pooling

How a network shrinks a picture by summarising small windows, keeping the strongest signal or the average of each one.

14 min readFree

Shrinking a picture while keeping what matters

The last lesson, Strides, ended with a goal: shrink a picture while keeping what matters. A bigger stride is one way to do it, because the filter stops at fewer places. This lesson is about the classic way, a step called pooling.

Think about a long report. You do not have time to read it all, so you read one headline for each paragraph. The report gets much shorter. You lose some detail, but you still know what it says. Pooling does this to a grid of numbers. It looks at small windows, writes one number for each window, and throws the rest away.

Key words

Feature map
The grid of numbers a filter produces. Each number says how strongly a pattern shows up at that spot.
Pooling
Shrinking a feature map by replacing each small window of numbers with one number that summarises it.
Window
The small square of cells that pooling looks at in one go. A 2x2 window covers four cells.
Stride
How many cells the window moves between one step and the next.
Max pooling
Keep the biggest number in each window.
Average pooling
Keep the mean of the numbers in each window: add them up and divide by how many there are.
Channel
One feature map out of a stack of them. A layer with 12 filters makes 12 channels. CS231n calls each one a depth slice.

What pooling does

Wikipedia describes pooling layers as reducing the dimensions of data by combining the outputs of neuron clusters at one layer into a single neuron in the next layer. A neuron here is just one number in the grid. So a cluster of nearby numbers goes in, and one number comes out.

CS231n, the Stanford course notes on convolutional networks, says why we want this. A pooling layer progressively reduces the spatial size of the representation, so the network has fewer parameters and less computation, and that also helps control overfitting. In plain words: a smaller grid means less work for every layer after it. Overfitting means the network memorises its training pictures instead of learning patterns that work on new ones.

Wikipedia adds an idea behind it all: the exact location of a feature is less important than its rough location relative to other features. If a network is looking for an eye in a face, it matters more that the eye is above the nose than that it sits at exactly pixel 41.

Max pooling and average pooling

Wikipedia names two common types: max pooling uses the maximum value of each local cluster of neurons in the feature map, while average pooling takes the average value.

  • Max pooling

    Keep the strongest signal. Picture a group of people talking at once and you write down only the loudest voice. If any cell in the window saw the pattern strongly, the window remembers it.

  • Average pooling

    Keep the mean. Picture a class and you write down its average mark. One star student barely moves it, and one weak student barely moves it either. The window remembers the general level.

  • Which one wins

    Both CS231n and Wikipedia say average pooling was often used in the past but has fallen out of favour, because max pooling has been shown to work better in practice.

CS231n also mentions L2-norm pooling as another option. Max and average are the two you will meet most.

Window size and stride

Pooling needs two settings, the size of the window and the stride. They work like the filter size and stride from the last lessons. The window slides across the feature map, one stop at a time, and each stop gives one output number.

CS231n says the most common form is a pooling layer with filters of size 2x2 applied with a stride of 2. That cuts the width in half and the height in half, and it discards 75% of the activations. An activation is just one number in the feature map. Each max is taken over 4 numbers, a little 2x2 region. Wikipedia describes the same very common form, and says that 2x2 tiles are commonly used.

The stride matters a lot here. Wikipedia points out that without a stride greater than 1, pooling would not shrink anything: the window would just shuffle along one cell at a time. The stride is what actually causes the shrinking.

The Keras library agrees on the usual setup. Its MaxPooling2D layer has a default pool size of (2, 2), and if you do not set the strides they default to the pool size. So a 2x2 window gets a stride of 2 unless you say otherwise. Its padding setting "valid" means no padding. CS231n says it is not common to pad the input for a pooling layer.

CS231n gives the output size. If the input is W1 wide, the window is F wide and the stride is S, then the output is W2 = (W1 - F)/S + 1 wide, and the same for height. Each channel is pooled on its own, so the number of channels does not change.

W2=W1−FS+1W_2 = \frac{W_1 - F}{S} + 1

In words: take the input width, subtract the window width, divide by the stride, and add 1. It is the same formula as for a convolution, with the padding left out.

Tip

CS231n names only two settings you will see often: F = 2 with S = 2, and, more rarely, F = 3 with S = 2, which it calls overlapping pooling because the windows share cells. It says bigger windows are too destructive. When in doubt, use 2x2 with stride 2.

Nothing to learn

A convolution filter has weights, and training adjusts them. Pooling has none. CS231n says the pooling layer introduces zero parameters since it computes a fixed function of the input. The function is simply max, or average. It does the same thing every time, so there is nothing for training to change.

This is why pooling is cheap. It adds no new numbers to learn. It only makes the grid smaller, which makes the layers after it lighter.

Run it: a 6x6 feature map

Here is pooling by hand. The code below builds a 6x6 feature map. The function slides a window of the given size over it with the given stride. For each stop it collects the numbers under the window and keeps either the biggest one or their mean. It prints the original map, then both results.

Python
fmap = [
    [1, 0, 6, 1, 0, 0],
    [0, 3, 1, 0, 2, 0],
    [2, 1, 0, 0, 0, 1],
    [0, 0, 9, 2, 1, 0],
    [1, 0, 0, 1, 7, 1],
    [0, 2, 1, 0, 0, 0],
]

def pool(grid, size, stride, how):
    n = (len(grid) - size) // stride + 1
    out = []
    for r in range(n):
        row = []
        for c in range(n):
            patch = [grid[r * stride + i][c * stride + j]
                     for i in range(size) for j in range(size)]
            row.append(max(patch) if how == "max" else sum(patch) / len(patch))
        out.append(row)
    return out

def show(title, grid):
    print(title)
    for row in grid:
        print("  " + " ".join(f"{v:5.2f}" if isinstance(v, float) else f"{v:2d}" for v in row))

show("feature map (6x6)", fmap)
show("max pooling 2x2, stride 2 (3x3)", pool(fmap, 2, 2, "max"))
show("average pooling 2x2, stride 2 (3x3)", pool(fmap, 2, 2, "avg"))

Output

feature map (6x6)
   1  0  6  1  0  0
   0  3  1  0  2  0
   2  1  0  0  0  1
   0  0  9  2  1  0
   1  0  0  1  7  1
   0  2  1  0  0  0
max pooling 2x2, stride 2 (3x3)
   3  6  2
   2  9  1
   2  1  7
average pooling 2x2, stride 2 (3x3)
   1.00  2.00  0.50
   0.75  2.75  0.50
   0.75  0.50  2.00

The output is 3x3 each time, because (6 - 2)/2 + 1 = 3. Check one window by hand. The top-left window covers 1, 0 on the first row and 0, 3 on the second. The biggest is 3, so max pooling writes 3. The sum is 4 and 4 divided by 4 is 1.00, so average pooling writes 1.00. The top-middle window covers 6, 1, 1, 0. Max pooling keeps 6. The average is 8 divided by 4, which is 2.00.

Now look at the middle of the map, where the 9 sits. Its window holds 0, 0, 9 and 2. Max pooling writes 9, the loud voice, the whole strength of that spot. Average pooling writes 2.75, because the 9 gets diluted by the small numbers next to it. Max pooling keeps the sharp peak. Average pooling smooths it into the surroundings.

The map went from 36 numbers to 9. That is a quarter, which matches CS231n: 75% discarded. The next code checks the output size for a few inputs, using the formula.

Python
def out_size(w, f, s):
    return (w - f) // s + 1

for w, f, s in ((6, 2, 2), (224, 2, 2), (7, 2, 2), (7, 3, 2)):
    n = out_size(w, f, s)
    print(f"input {w}x{w}, window {f}x{f}, stride {s}: output {n}x{n}")
print("share of cells kept:", out_size(224, 2, 2) ** 2 / 224 ** 2)

Output

input 6x6, window 2x2, stride 2: output 3x3
input 224x224, window 2x2, stride 2: output 112x112
input 7x7, window 2x2, stride 2: output 3x3
input 7x7, window 3x3, stride 2: output 3x3
share of cells kept: 0.25

A 224x224 picture becomes 112x112, an example CS231n uses too. Only a quarter of the cells are left. The 7x7 case shows what happens when the window does not fit evenly. The last row and column are never covered, so they are simply left out. The size formula in Keras rounds down the same way, with math.floor.

  1. 1

    Slide the window

    Start at the top-left corner. Take the cells under the window.

  2. 2

    Summarise

    Keep the biggest number for max pooling, or the mean for average pooling.

  3. 3

    Jump by the stride

    Move the window 2 cells along. At the end of a row, go down 2 rows and start again at the left.

  4. 4

    Collect

    The summaries, in order, make the new smaller grid.

Small shifts matter less

Wikipedia says the pooling operation grants a degree of local translational invariance to the features contained in it, which makes the network more robust to variations in their positions. Translational invariance is a long phrase for a simple idea: if a pattern moves a little, the answer stays about the same.

Wikipedia is also careful here. It says most convolutional networks are not strictly translation-invariant, and that pooling alone does not give global translation invariance unless a form of global pooling is used. So treat this as a small helping effect, not a promise.

Let us see the small effect. The code below takes the same 6x6 map, moves every row one cell to the right (the first column fills with zeros, and the last column falls off the edge), and max pools both versions. Then it counts what stayed the same.

Python
def max_pool(grid):
    return [[max(grid[r + i][c + j] for i in range(2) for j in range(2))
             for c in range(0, 6, 2)] for r in range(0, 6, 2)]

fmap = [
    [1, 0, 6, 1, 0, 0],
    [0, 3, 1, 0, 2, 0],
    [2, 1, 0, 0, 0, 1],
    [0, 0, 9, 2, 1, 0],
    [1, 0, 0, 1, 7, 1],
    [0, 2, 1, 0, 0, 0],
]

# move every row one cell to the right; the new left column is zeros
shifted = [[0] + row[:-1] for row in fmap]

def show(title, grid):
    print(title)
    for row in grid:
        print("  " + " ".join(f"{v:2d}" for v in row))

a, b = max_pool(fmap), max_pool(shifted)
show("shifted map", shifted)
show("max pooled, original", a)
show("max pooled, shifted", b)

cells = [(x, y) for ra, rb in zip(a, b) for x, y in zip(ra, rb)]
print("cells identical:", sum(x == y for x, y in cells), "of 9")
print("biggest change in any cell:", max(abs(x - y) for x, y in cells))
print("strongest three, original:", sorted((v for r in a for v in r), reverse=True)[:3])
print("strongest three, shifted: ", sorted((v for r in b for v in r), reverse=True)[:3])

Output

shifted map
   0  1  0  6  1  0
   0  0  3  1  0  2
   0  2  1  0  0  0
   0  0  0  9  2  1
   0  1  0  0  1  7
   0  0  2  1  0  0
max pooled, original
   3  6  2
   2  9  1
   2  1  7
max pooled, shifted
   1  6  2
   2  9  2
   1  2  7
cells identical: 5 of 9
biggest change in any cell: 2
strongest three, original: [9, 7, 6]
strongest three, shifted:  [9, 7, 6]

Every strong peak survived. The 9, the 7 and the 6 all land in the same output cells as before. The strongest three values are the same list for both maps. Five of the nine cells are identical, and the biggest change in any cell is 2, on a scale where the peaks are 6, 7 and 9.

Why did four cells change at all? A value that sits on the edge of a window can slide across the border into the next window. Then the first window loses it and the second window gains it. That is also the limit of the effect: a one-cell shift only stays hidden when the number stays inside its window. A bigger shift would move more values across borders.

Global average pooling

So far the window was small. Wikipedia mentions another kind: local pooling combines small clusters, while global pooling acts on all the neurons of the feature map. The window is the whole map. Keras has a layer for it called GlobalAveragePooling2D. Global average pooling takes one feature map and gives one number, the mean of every cell in it. A stack of 12 channels becomes 12 numbers.

Python
fmap = [
    [1, 0, 6, 1, 0, 0],
    [0, 3, 1, 0, 2, 0],
    [2, 1, 0, 0, 0, 1],
    [0, 0, 9, 2, 1, 0],
    [1, 0, 0, 1, 7, 1],
    [0, 2, 1, 0, 0, 0],
]
total = sum(sum(row) for row in fmap)
print("global average pooling of one 6x6 map:", total, "/ 36 =", round(total / 36, 2))
print("global max pooling of the same map:", max(max(row) for row in fmap))

Output

global average pooling of one 6x6 map: 43 / 36 = 1.19
global max pooling of the same map: 9

The 6x6 map adds up to 43, and 43 divided by 36 cells is 1.19. The map has been squeezed to a single number. The global max, for comparison, is just the strongest cell, 9.

Do we still need pooling?

Not everyone likes it. CS231n says many people dislike the pooling operation and think we can get away without it. It points to a paper called Striving for Simplicity: The All Convolutional Net, which proposes to discard the pooling layer in favor of an architecture that only consists of repeated convolution layers, using a larger stride once in a while to shrink the output. That is the trick from the Strides lesson.

Wikipedia notes a recent trend towards using smaller filters or discarding pooling layers altogether. CS231n adds that dropping pooling has been found to be important in training good generative models, and says it seems likely that future architectures will feature very few to no pooling layers.

Knowing both ways, pooling and a bigger stride, lets you read either kind of network.

Quick checks

Quick check

A feature map is 8 wide. You apply 2x2 pooling with stride 2. How wide is the output?

Quick check

A window holds 1, 0, 8 and 3. What do max pooling and average pooling write?

Quick check

How many weights does a pooling layer learn?

What to remember

  • What it does

    Pooling shrinks a feature map by replacing each small window with one number. The usual setting is a 2x2 window with stride 2, which keeps a quarter of the cells.

  • Max or average

    Max pooling keeps the strongest signal and has been shown to work better in practice. Average pooling keeps the mean and has fallen out of favour.

  • No weights

    Pooling learns nothing. Small shifts in position matter a little less afterwards, and some designs now shrink with a bigger stride instead.

Next: the convolutional neural network, putting the pieces together.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.