esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Machine Learning Foundations / Advanced / 03.28

Padding: keeping the edges

Why a picture shrinks every time a filter slides over it, how a frame of zeros fixes that, and the size formula, with code you can run.

14 min readFree

Keeping the edges of a picture

The last lesson, Convolution, ended with a promise: padding, keeping the edges of a picture. This is that lesson.

Picture a photo you want to scan with a small square magnifying glass. You slide the glass along the first row, then the next row, and so on. There is one rule: the whole glass must sit on the photo. It may not hang over the edge. Now ask: where can the glass sit? Not at the very edge, because half of it would hang off. So the glass can only visit spots a little way in. The list of spots is shorter than the number of pixels in the photo.

That small fact causes two real problems in a convolutional network. The picture shrinks a little at every layer, and the pixels at the edge get looked at less than the ones in the middle. Padding is the cheap fix for both. It is one of the first settings you meet in any convolution layer.

Key words

Filter
A small grid of weights, such as 3 × 3, that slides over the picture. Also called a kernel.
Convolution
Sliding the filter over the picture. At each spot, multiply the filter's numbers by the pixels under it and add them up. Each spot gives one number in the output.
Output
The new grid the filter makes, one number per spot it visited. Wikipedia calls it a feature map.
Padding
Extra pixels added around the border of the picture before the filter slides over it.
Zero padding
Padding where every added pixel is 0. A frame of zeros.
"valid" and "same"
The two usual padding settings. "valid" means no padding. "same" means pad just enough that the output is as big as the input.

Problem one: the picture shrinks

Take a tiny 6 × 6 picture and a 3 × 3 filter. The filter can start in any of 4 places along a row, because the 3 pixels under it must fit: positions 1 to 3, 2 to 4, 3 to 5 and 4 to 6. The same is true going down. So the output is 4 × 4. We put 36 pixels in and get 16 numbers out.

Now stack layers, the way a deep network does. The 4 × 4 output becomes the input of the next layer. Another 3 × 3 filter turns it into 2 × 2. Let us count with code.

Python
size = 6
layer = 0
print("layer", layer, "picture is", size, "x", size)
while size - 3 + 1 >= 1:
    size = size - 3 + 1   # a 3x3 filter, no padding
    layer = layer + 1
    print("layer", layer, "picture is", size, "x", size)
print("one more layer would not fit: the filter is bigger than the picture")

Output

layer 0 picture is 6 x 6
layer 1 picture is 4 x 4
layer 2 picture is 2 x 2
one more layer would not fit: the filter is bigger than the picture

After two layers the picture is 2 × 2, and a third 3 × 3 filter cannot fit at all. Every layer with a 3 × 3 filter loses 2 pixels of width and 2 of height, so a network with dozens of layers would run out of picture.

Stanford's CS231n notes name the same worry. If the layers only do "valid" convolutions, they write, the size of the volumes would reduce by a small amount after each layer.

Problem two: the edges are seen less

There is a second, quieter problem. In the middle of a picture, a pixel sits under the filter many times, once for every spot where the filter covers it. A pixel in the corner is covered only once, because the filter can reach the corner in one spot only.

Wikipedia puts it this way: padding is done so that the border pixels are not undervalued (lost) from the output, because they would ordinarily participate in only a single receptive field instance. A receptive field is the patch of the picture one output number looks at. So a pixel in the corner only gets one say.

Let us count it. For a 6 × 6 picture and a 3 × 3 filter with no padding, this code counts, for each pixel, how many filter spots cover it.

Python
n = 6
f = 3
for row in range(n):
    line = []
    for col in range(n):
        looks = 0
        for top in range(n - f + 1):
            for left in range(n - f + 1):
                if top <= row < top + f and left <= col < left + f:
                    looks += 1
        line.append(looks)
    print(line)

Output

[1, 2, 3, 3, 2, 1]
[2, 4, 6, 6, 4, 2]
[3, 6, 9, 9, 6, 3]
[3, 6, 9, 9, 6, 3]
[2, 4, 6, 6, 4, 2]
[1, 2, 3, 3, 2, 1]

Read it like a map of attention. The four middle pixels are looked at 9 times each. The four corner pixels are looked at once. The strip along the edge sits in between. Something near the border gets far fewer chances to be noticed.

CS231n adds a second reason to pad: they say it actually improves performance, because without it, the information at the borders would be "washed away" too quickly.

The fix: a frame of zeros

The fix is simple. Before the filter starts sliding, put a frame around the picture. Like a photo in a picture frame, the frame is made of new pixels, and every new pixel is 0. This is zero padding.

Why zeros? Multiplying by zero adds nothing to the sum, so the frame invents no new picture. It only gives the filter more spots to stand on.

CS231n describes the benefit: zero padding lets us control the spatial size of the output. Most often it is used to exactly preserve the spatial size of the input, so the input and output width and height are the same. Wikipedia says the same thing in its own words.

  1. 1

    1. Choose the padding p

    p is how many pixels of zeros go on each side. p = 1 means one ring of zeros all the way around.

  2. 2

    2. Build the frame

    A 5 × 5 picture with p = 1 becomes 7 × 7: 5 plus 1 on the left plus 1 on the right.

  3. 3

    3. Slide as usual

    Run the filter over the bigger, framed grid. Nothing else about the convolution changes.

  4. 4

    4. Count the spots

    A 3 × 3 filter now fits in 5 places along a row of 7 pixels, so the output is 5 × 5, the same as the original.

Here it is in code. The picture holds the numbers 1 to 25. The filter is a plus shape: 1 in the middle and on the four sides, 0 in the corners. At each spot the code multiplies and adds, as before. The code prints the padded grid, the output with padding, and the output with no padding, for comparison.

Python
image = [
    [1, 2, 3, 4, 5],
    [6, 7, 8, 9, 10],
    [11, 12, 13, 14, 15],
    [16, 17, 18, 19, 20],
    [21, 22, 23, 24, 25],
]
filt = [
    [0, 1, 0],
    [1, 1, 1],
    [0, 1, 0],
]

def pad_zeros(grid, p):
    width = len(grid[0]) + 2 * p
    framed = [[0] * width for _ in range(p)]
    for row in grid:
        framed.append([0] * p + row + [0] * p)
    framed += [[0] * width for _ in range(p)]
    return framed

def convolve(grid, k):
    f = len(k)
    size = len(grid) - f + 1
    out = []
    for top in range(size):
        out_row = []
        for left in range(size):
            total = 0
            for i in range(f):
                for j in range(f):
                    total += grid[top + i][left + j] * k[i][j]
            out_row.append(total)
        out.append(out_row)
    return out

padded = pad_zeros(image, 1)
print("padded grid:")
for row in padded:
    print(row)
print("output:")
for row in convolve(padded, filt):
    print(row)
print("without padding, output:")
for row in convolve(image, filt):
    print(row)

Output

padded grid:
[0, 0, 0, 0, 0, 0, 0]
[0, 1, 2, 3, 4, 5, 0]
[0, 6, 7, 8, 9, 10, 0]
[0, 11, 12, 13, 14, 15, 0]
[0, 16, 17, 18, 19, 20, 0]
[0, 21, 22, 23, 24, 25, 0]
[0, 0, 0, 0, 0, 0, 0]
output:
[9, 13, 17, 21, 19]
[25, 35, 40, 45, 39]
[45, 60, 65, 70, 59]
[65, 85, 90, 95, 79]
[59, 83, 87, 91, 69]
without padding, output:
[35, 40, 45]
[60, 65, 70]
[85, 90, 95]

Look at the sizes. A 5 × 5 picture went in. With the frame, a 5 × 5 output came out. Without it, only a 3 × 3 output came out. Check the top-left number by hand: the plus filter, standing on the top-left corner of the padded grid, sees 0 above, 0 on the left, 1 in the middle, 2 on the right and 6 below. Add them: 0 + 0 + 1 + 2 + 6 = 9. That matches the first number of the output. The corner pixel 1 now takes part in that sum, which it never did without the frame.

"valid" and "same" in plain words

Libraries rarely ask you to type the number of padding pixels. They give you two named choices instead. The Keras documentation for its Conv2D layer describes the padding setting like this: it is a string, either "valid" or "same". "valid" means no padding. "same" results in padding evenly to the left/right or up/down of the input. It adds that when padding="same" and strides=1, the output has the same size as the input.

The TensorFlow documentation for Conv2D uses the same words. The Keras signature shows padding="valid" as the default, so if you do not choose, you get no padding.

A way to remember it: "valid" keeps only the spots where the filter fits fully inside the picture. "same" frames the picture so the output stays the same size. Notice the phrase "strides=1". The filter in this lesson moves one pixel at a time. When it jumps further, the output shrinks even with "same". Strides are the next lesson.

How much padding does "same" use? For a filter of width f, both CS231n and Wikipedia say that setting the zero padding to p = (f − 1) / 2 when the stride is 1 keeps the input and output the same size. For a 3 × 3 filter, p = (3 − 1) / 2 = 1, which is the one ring of zeros we used above. For a 5 × 5 filter, p = 2.

p=f−12p = \frac{f - 1}{2}

In words: the padding on each side is the filter width minus one, divided by two. That only gives a whole number when f is odd, which is one reason filters are usually 3 × 3, 5 × 5 or 7 × 7.

The output-size formula

You can work out the output size without running anything. Call the picture width n, the filter width f and the padding p, with the stride at 1. CS231n writes the general formula as (W − F + 2P) / S + 1, where S is the stride. Wikipedia gives the same one. Put S = 1 and use our letters.

nout=n−f+2p+1n_{\text{out}} = n - f + 2p + 1

In words: start with the picture width, take away the filter width, add the padding twice because there is a frame on both sides, then add one.

Test it on the 6 × 6 picture from the start. No padding gives 6 − 3 + 0 + 1 = 4. One ring of padding gives 6 − 3 + 2 + 1 = 6. Both match what we saw. CS231n has its own example: a 7 × 7 input and a 3 × 3 filter, with stride 1 and no padding, gives a 5 × 5 output.

The code below prints the output size for pictures from 5 up to 10 pixels wide, with and without padding. Then it checks one case, n = 8, against a real slide of a 3 × 3 filter over a grid of ones, so it counts real output numbers rather than trusting the formula.

Python
def out_size(n, f, p):
    return n - f + 2 * p + 1   # stride 1

f = 3
print("n  no padding  padding 1")
for n in range(5, 11):
    print(f"{n:<2} {out_size(n, f, 0):<11} {out_size(n, f, 1)}")

# Check n = 8 against a real slide of a 3x3 filter over a zero-framed grid.
n = 8
grid = [[1] * n for _ in range(n)]
framed = [[0] * (n + 2)] + [[0] + row + [0] for row in grid] + [[0] * (n + 2)]
kernel = [[1] * 3 for _ in range(3)]

def slide(g):
    size = len(g) - 3 + 1
    return [[sum(g[t + i][l + j] * kernel[i][j] for i in range(3) for j in range(3))
             for l in range(size)] for t in range(size)]

print("real output, no padding:", len(slide(grid)), "x", len(slide(grid)[0]))
print("real output, padding 1: ", len(slide(framed)), "x", len(slide(framed)[0]))
print("formula says:", out_size(8, 3, 0), "and", out_size(8, 3, 1))

Output

n  no padding  padding 1
5  3           5
6  4           6
7  5           7
8  6           8
9  7           9
10 8           10
real output, no padding: 6 x 6
real output, padding 1:  8 x 8
formula says: 6 and 8

Without padding, the output is always 2 smaller than the input. With padding 1, it is exactly as big. The real slide for n = 8 gives 6 × 6 and 8 × 8, as the formula says.

Other kinds of padding

Zero is the usual choice, but it is not the only one. Wikipedia says you can pad with zeros (or other values, such as the average of the region), and it shows three more ways to fill the frame. Replication copies the closest pixel inside. Reflection mirrors the pixels inside across the border of the image. Circular padding wraps around to the other side of the picture.

Why pad? Three reasons

  • The picture keeps its size

    With the right padding, each layer hands the next one a grid as big as the one it received. You can stack many layers without running out of picture.

  • The edges count

    The pixels in the corners and along the border now sit under the filter more than once, so they take part in the output. Edges are not lost.

  • The sizes are easy to plan

    CS231n says that if you do not pad, you must carefully keep track of the input volumes through the whole network and make sure all the strides and filters work out. Padding takes away that headache.

Check yourself

Quick check

A 10 × 10 picture goes through a 3 × 3 filter with stride 1 and no padding. How big is the output?

Quick check

You want a 5 × 5 filter to keep the picture the same size, with stride 1. How many pixels of zeros go on each side?

Quick check

Why do corner pixels get looked at less when there is no padding?

Recap

  1. 1

    1. The problem

    Without padding, every convolution shrinks the picture, and edge pixels are looked at less than middle ones.

  2. 2

    2. The fix

    Put a frame of zeros around the picture. Zero adds nothing to the sums, but gives the filter room.

  3. 3

    3. The words

    "valid" means no padding. "same" pads so that, with stride 1, the output is as big as the input.

  4. 4

    4. The formula

    Output size is n − f + 2p + 1. For "same" with stride 1, p = (f − 1) / 2.

Next: strides, how far the filter jumps.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.