Definition
Plain language
A grader that scores each step of an AI's reasoning, not just the final answer.
As stated in the literature
A reward model trained to score intermediate reasoning steps rather than only outcomes, providing dense supervision for RL; typically expensive because it requires step-level annotation.
Also called: PRM, process reward models
Why it matters: By rewarding good reasoning steps, not just correct endings, it gives much richer guidance during training, at the cost of expensive step-by-step labeling.
For example, instead of only checking the final answer to a math problem, it grades each line of the working to catch a mistake midway.
Heard on the show
“This paper sits in a longer arc that includes process reward models in math reasoning — the "Let's Verify Step by Step" line of work — and a broader move toward dense step-level supervision.”Episode 008 — Why Long-Horizon AI Agents Get Stuck, and a Milestone-Based Fix That Helps