Definition
Plain language
In AI training, a score for how much better a particular attempt turned out than the model's average attempt.
As stated in the literature
In policy-gradient RL, the difference between an action's return and a baseline estimate of expected return; group-relative methods like GRPO compute it by comparing rollouts on the same prompt rather than learning a separate value function.
Also called: advantages
Why it matters: It tells a training algorithm which attempts to reinforce and which to discourage, so the model learns from its own better-than-average tries rather than treating all outcomes the same.
For example, if a model's typical attempt at a problem scores 4 out of 10 but one particular attempt scores 8, that attempt's advantage is the gap that tells training to do more of what it did.
Heard on the show
“And the cost advantage is what makes the paper's deeper point.”Episode 005 — Why a Debugger Designed for Humans Is the Wrong Tool for an AI Agent