Definition
Plain language
A math safety net that stops fixed-point training from drifting into unstable regions, because the gradient itself blows up if it tries.
As stated in the literature
In Attractor Models, the observation that the implicit gradient involves an inverse of one-minus-Jacobian which diverges as the refinement step loses contractivity, structurally confining gradient descent to convergent regimes.
Why it matters: It gives fixed-point models a built-in stability mechanism, so training can't easily wander into regimes where the model wouldn't actually converge at inference.
For example, if a refinement step starts losing its contractive behavior, the inverse term in the implicit gradient blows up and training is pushed back toward parameters where convergence is well-behaved.
Heard on the show
“They call it the implicit gradient barrier.”Episode 041 — When the Iteration Teaches the Model to Skip the Iteration