Definition
Plain language
Processing a sequence step by step, with the model carrying a hidden state from one step to the next.
As stated in the literature
A computational pattern that maintains and updates a state across sequence positions, contrasted with parallel attention-based processing.
Also called: recurrence, recurrences, recurrent state
Why it matters: Recurrent processing trades parallelism for constant-size memory per step, making it attractive whenever sequence length explodes beyond what attention can handle.
For example, an old-school text model reads one word, updates a memory vector, reads the next word, updates the vector again, and so on through the sentence.
Heard on the show
“"Demystifying the recurrence" kind of reveals the recurrence isn't doing much recurring.”Episode 141 — How Two Tokens Reopened a Reasoning Method the Field Had Given Up On