Series
math-behind-neural-networks
6 parts, in order.
- 01
Aug 18, 2026
The Math Behind Neural Networks, Part 1.4 — Which Loss to Choose, and Why
The output distribution was a choice. Other choices give other costs, each predicting a different statistic — and why cross-entropy beats squared error when the gradient would otherwise fall silent.
- 02
Aug 2, 2026
The Math Behind Neural Networks, Part 1.3 — Where the Loss Function Comes From
Cost functions derived from maximum likelihood. Why probabilities multiply, how the logarithm keeps the order— the top-down route to the loss we assumed in 1.1.
- 03
Jul 25, 2026
The Math Behind Neural Networks, Part 0 — Where Deep Learning Comes From
What a neural network is before any of the math: why deep learning had to exist, what every term actually means, and why a model must learn its own features. The one model the whole field grows from, built from zero.
- 04
Jun 9, 2026
The Leash That Makes AI Polite
How a 1951 statistics formula quietly keeps your chatbot from going feral. KL divergence, RLHF, and the tension between them.
- 05
May 24, 2026
The Math Behind Neural Networks, Part 1.2 — The Mathematics of Learning
The derivative, derived by hand from MSE, and gradient descent built on top of it. Where the 2/n comes from, why we subtract the gradient, and a full worked training example.
- 06
May 9, 2026
The Math Behind Neural Networks, Part 1.1 — The Mathematical Foundations
Slope, linear regression, and the error function, built from the ground up. Why we divide by n, why we square the error, and why a loss value alone cannot tell you how to improve.