Definition
Plain language
A basic calculus rule for combining how one change leads to another through a chain of steps.
As stated in the literature
In information theory, Shannon's chain rule decomposes joint entropy into a sum of conditional and marginal entropies; in calculus, the standard rule for differentiating function compositions.
Also called: Shannon's chain rule
Why it matters: It's the mathematical backbone of backpropagation, without which training neural networks would be infeasible.
For example, if a model's loss depends on a parameter through several layers, the chain rule multiplies the sensitivities at each layer to get the overall gradient.
Heard on the show
“You compute it with the chain rule.”Episode 025 — The Missing Gradient Term That Predicts Sycophancy in RLHF