Definition
Plain language
Running a neural network's layers multiple times over the same input to get more computation out of the same weights.
As stated in the literature
An architectural pattern that loops a stack of blocks repeatedly during a forward pass, trading sequential compute for additional reasoning capacity at fixed parameter count; instantiated in Universal Transformers, looped transformers, and attractor-style models.
Also called: recurrent depth, depth-recurrence
Why it matters: It offers a way to spend more compute on harder problems at inference time without making the model itself bigger.
For example, a model with six layers runs them in a loop four times on a hard problem, giving it twenty-four effective layers of computation with the parameter count of six.
Heard on the show
“It's a depth-recurrence story dressed in neuroscience clothing.”Episode 085 — Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction