Definition
Plain language
The way you train a network that reads a sequence step by step — by unrolling all the steps and tracing each mistake backward through the whole chain.
As stated in the literature
BPTT — applying backpropagation to a recurrent computation by unrolling it across time steps and propagating gradients through every step; exact but expensive for long sequences, which motivates approximations like truncated backpropagation.
Also called: BPTT, backprop through time
Why it matters: It is how sequence-reading networks actually learn from their mistakes, though its cost on long sequences is what forces shortcuts like truncation.
For example, to train a network that predicts the next word in a sentence, you unroll each word position and trace every prediction error back through the whole sentence.
Heard on the show
“And the obvious fallback — unroll the sequence and backprop through time — is computationally a non-starter at any reasonable length.”Episode 032 — A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking