Definition
Plain language
A function that pulls things closer together each time you apply it, so iteration converges.
As stated in the literature
A property of a map under which repeated application contracts distances, guaranteeing convergence to a fixed point.
Why it matters: Contractive maps are the mathematical foundation for many iterative algorithms used in optimization, equation solving, and equilibrium-style neural networks.
For example, repeatedly averaging a number with zero halves it each time, contracting toward a fixed point of zero.
Heard on the show
“Implicit fixed-point methods need the recurrence to be contractive, which this isn't.”Episode 032 — A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking