Definition
Plain language
A single number measuring how strong the learning signal is at a given training step.
As stated in the literature
The magnitude of the gradient vector during training; a diagnostic for learning-signal strength, where anomalously small values can reveal silent bugs (as in the DeepSpeed CPU-offload case) or low-variance reward groups.
Also called: gradient norms
Why it matters: It's a quick health check on whether a model is actually learning, helping catch silent failures that would otherwise waste a training run.
For example, if the learning signal suddenly reads near zero when it shouldn't, that tiny gradient norm can tip you off to a hidden bug in the training setup.
Heard on the show
“The buggy run shows substantially smaller gradient norms throughout training, because there's just less signal flowing into each step.”Episode 009 — How Two Silent Library Bugs Quietly Invalidated a Wave of Reasoning Papers