Definition
Plain language
A training trick that starts with a short backward chain and slowly lengthens it as the model stabilizes.
As stated in the literature
In HRM-Text, a curriculum that begins with a short truncated-backprop horizon (e.g., 2 steps) and gradually extends it across training to stabilize gradient flow through recurrent unrolls.
Why it matters: Without such a curriculum, deep recurrent models often blow up early in training and never reach their potential.
For example, training might start by backpropagating only 2 steps back, then gradually extend to 16 steps as the model stops diverging.
Heard on the show
“… There's a second trick they layer on top — they call it warmup deep credit assignment — where they start training with an even shorter backward horizon, just two steps, and …”Episode 074 — How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning