Definition
Plain language
A foundational rule in reinforcement learning for valuing a situation by the rewards you can still expect from it.
As stated in the literature
Richard Bellman's recursive relation expressing the value of a state as immediate reward plus the discounted value of successor states; the formal root of rewards-to-go and dynamic programming in RL.
Also called: Bellman
Why it matters: It is the recursive backbone that lets reinforcement-learning agents assign value to situations and plan toward long-term reward.
For example, it says the worth of being one move from a goal equals the immediate reward plus the value of wherever that move lands you.
Heard on the show
“The standard answer goes all the way back to Bellman in the fifties — it's called rewards-to-go.”Episode 160 — Training an AI to Take Its Own Notes, So Its Future Self Works Better