Definition
Plain language
A Transformer variant that runs the same layers over and over to think harder, rather than stacking more layers.
As stated in the literature
A 2018 architecture that applies a single Transformer block recurrently across depth, sharing weights across iterations; foundational reference for depth-recurrent and looped sequence models.
Why it matters: It introduced the idea that adaptive depth — thinking more on harder inputs — can substitute for stacking more parameters.
For example, a Universal Transformer might run its single layer five times on an easy input and twenty times on a hard one to think longer.
Heard on the show
“Universal Transformers are the canonical reference, there's been work on equilibrium models, and more recently Ouro, the model they use here, is itself a looped architecture.”Episode 085 — Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction