Definition
Plain language
How much better or worse a situation turned out than expected after one step — the surprise that drives learning.
As stated in the literature
TD error — the difference between the estimated value of a state and the value of the state actually reached after an action; used in G2PO as an edge-level advantage measuring how much progress a single move made.
Also called: TD error, temporal difference
Why it matters: It is the basic learning signal that lets an agent figure out which actions made things better or worse, step by step.
For example, if a chess move you expected to be neutral suddenly puts you in a winning position, that gap between expectation and reality is the surprise the error captures.
Heard on the show
“If you've heard the term temporal-difference error, that's exactly what this is: not how high up you are, but how much altitude this one step gained you.”Episode 165 — A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants