Definition
Plain language
Judging an action not by what happened right after it, but by all the good that followed from it down the line.
As stated in the literature
A credit-assignment formulation, dating to Bellman, that values an action by the sum of future rewards downstream of it rather than the immediate next reward; the basis for crediting good early notes by later-task success in long-lifecycle agents.
Also called: reward-to-go
Why it matters: It lets an agent value early actions by their long-run payoff, which is essential when good decisions only pay off much later.
For example, a chess opening move gets credit not for the next move alone but for the whole winning game that flows from it.
Heard on the show
“The standard answer goes all the way back to Bellman in the fifties — it's called rewards-to-go.”Episode 160 — Training an AI to Take Its Own Notes, So Its Future Self Works Better