Definition
Plain language
The step-by-step process a neural network uses to trace each mistake backward through its layers and figure out how to adjust.
As stated in the literature
The algorithm that computes gradients of a loss with respect to every parameter by applying the chain rule backward through the network's layers; the foundation of training for essentially all modern neural networks, and the metaphor behind tracing blame backward through a multi-agent system.
Also called: backprop
Why it matters: It is the core mechanism that lets neural networks actually learn from their mistakes, and without it modern AI training would not work.
For example, if a network labels a photo of a cat as a dog, backpropagation traces that error back through every layer to nudge each internal setting slightly toward the correct answer.
Heard on the show
“That's just how backpropagation works — the error at the output gets passed back down through every floor.”Episode 193 — Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer