Training
LeNet-5 with the original layer shapes, swapping tanh and average pooling for ReLU and max
pooling. Trained from scratch in PyTorch on MNIST with Adam and mild affine jitter, on the
Apple silicon GPU through the mps backend. 15 epochs, about 90 seconds, 99.30% on
the 10,000 held-out test images.
What those two swaps were actually worth, measured across four seeds