Recurrent-Depth Models

Recurrent-depth models are transformer architectures that reuse a shared block of layers in a loop, repeatedly updating a continuous hidden state rather than passing activations through a fixed stack of distinct layers. This lets the model allocate a variable, test-time-adjustable amount of internal “thinking” per input, and their continuous, iterated dynamics make it possible to study the model's computation as a genuine dynamical system exhibiting things like chaos and slow saddle-point transients.