Definition
Plain language
A reinforcement-learning trick that heavily rewards the rare successful attempts when most attempts fail.
As stated in the literature
An RL advantage-weighting scheme that exponentially up-weights rare high-reward rollouts so sparse successes aren't washed out by averaging; suited to low-pass-rate tasks.
Why it matters: It keeps learning alive on very hard tasks where success is rare, so the model still picks up signal instead of stalling.
For example, if a model solves only one out of a hundred attempts at a hard problem, this method makes that single success count heavily instead of being drowned out.
Heard on the show
“The algorithm they use there is called entropic advantage weighting, and the one-sentence version is: exponentially up-weight the rare wins so they don't get buried in the average.”Episode 088 — Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough