Definition
Plain language
A cheap way to train math reasoning into a model by editing only the few tokens where it actually matters.
As stated in the literature
A post-training method that uses entropy-gated contrastive supervision on a small set of high-uncertainty token positions in failed rollouts, reproducing RL-style reasoning gains at roughly three orders of magnitude lower cost.
Why it matters: If a cheap surgical edit at a few critical tokens can match expensive RL, it suggests reasoning gains live in a tiny fraction of decisions and reshapes how teams should budget post-training compute.
For example, instead of running expensive full reinforcement learning, the method picks out the handful of word positions in a failed attempt where the model was most uncertain and just trains on those.
Heard on the show
“Which they call ReasonMaxxer.”Episode 026 — What RL Actually Does to Language Models, at the Token Level