Definition
Plain language
An estimate of how good your situation is right now, in terms of the total reward you can expect from here on.
As stated in the literature
A learned estimate of expected cumulative future reward from a given state (or state-action pair); in Agentic Monte Carlo, a soft value function trained by offline regression on base-model rollouts serves as the resampling critic.
Also called: soft value function, value model
Why it matters: It lets an agent judge whether a situation is promising without having to play out every possible future.
For example, in a chess game the value function estimates how likely you are to win from the current board position.
Heard on the show
“Those leftovers don't telescope away, and they depend on the value function, which you cannot read off the two models.”Episode 173 — The Free Step-Level Grader Hiding in Every RL Training Run