Definition
Plain language
Generating one word at a time, with each word depending on what came before.
As stated in the literature
A generation regime where the model predicts each token conditional on the sequence of previously generated tokens.
Also called: autoregressively
Why it matters: It's the generation pattern behind every mainstream LLM and shapes both their strengths — coherent text — and their weaknesses, like compounding errors.
For example, an autoregressive model writes a sentence one word at a time, with each new word chosen given everything it's written so far.
Heard on the show
“Camp one, autoregressive drafters: the junior writes one word at a time, each seeing the ones before it.”Episode 179 — How DeepSeek Made One User Faster Without Slowing Down the Crowd