ArchitectureMathematical
◆ The PatternExploiting spatial structure with shared local filters
Convolutional layers apply a small learnable filter across the entire input — translational equivariance.
(f * g)[i,j] = Σₘ Σₙ f[m,n] · g[i−m, j−n]
f = filter/kernel | g = input | slide filter, compute dot product at each position
Output size = ⌊(W − K + 2P)/S⌋ + 1
W = input width | K = kernel size | P = padding | S = stride
// Convolution operation — kernel sliding over input
Kernel Size3×3
Stride1
Pattern bridge: Sliding a kernel across an image is the same operation as a moving average sliding across a price series. Both detect local patterns through shared weights.
Worked example
A 224×224 input with a 3×3 kernel, padding 1 and stride 2 gives a 112×112 output. Mapping 64 channels to 128 takes 73,856 parameters, because the same small filter is reused at every position. A fully connected layer between the same two shapes would need about 5×10¹². Weight sharing is what makes images tractable.
Where it misleads
- Convolutions are translation-equivariant, not invariant; pooling and training add some invariance, and only to small shifts.
- A small receptive field sees only local patterns; depth or dilation is needed for global context.
- Vision transformers match or beat CNNs at scale; the convolution’s built-in assumption is a help with little data, not a law.
In code
out = lambda W, K, P, S: (W - K + 2 * P) // S + 1 size = out(224, 3, 1, 2) # 224x224 input, 3x3 kernel, padding 1, stride 2 conv_params = 3 * 3 * 64 * 128 + 128 # 64 -> 128 channels, shared across positions dense_params = (224 * 224 * 64) * (112 * 112 * 128) # a fully connected layer between the same shapes
Assumes import numpy as np. Each snippet carries its own example numbers; the comments say which are made up or simulated.
The same idea with a library
self.conv1 = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1) self.pool = nn.MaxPool2d(kernel_size=2, stride=2) self.conv2 = nn.Conv2d(64, 128, kernel_size=3, padding=1)
Sources
- Y. LeCun, L. Bottou, Y. Bengio & P. Haffner, “Gradient-Based Learning Applied to Document Recognition”, Proceedings of the IEEE 86(11), 1998
- V. Dumoulin & F. Visin, “A Guide to Convolution Arithmetic for Deep Learning”, arXiv:1603.07285, 2016
Last reviewed October 2026 · How this site checks its content · Report a mistake
Linked from 1