Definition
Plain language
An approximation that's accurate when the thing you're ignoring is small.
As stated in the literature
A Taylor-style approximation accurate to leading order in a small parameter; in the Attractor Models paper, the implicit gradient approximation has error scaling as α² where α is the blend coefficient.
Also called: first-order accurate
Why it matters: Many practical training tricks only work because higher-order error terms are negligible, and knowing when that holds tells you when the trick is safe.
For example, treating the step size as small lets you ignore quadratic terms and write the update as a simple linear correction.
Heard on the show
“This is a first-order method on small sets, and the authors say as much — they're upfront that gradients here might miss the complicated nonlinear ways agents tangle together over long workflows.”Episode 181 — How to Backpropagate Blame Through a Team of Chatbots — And When It Backfires