Definition
Plain language
A grader that scores each step of an AI's reasoning, not just the final answer.
As stated in the literature
A reward model trained to score intermediate reasoning steps rather than only outcomes, providing dense supervision for RL; typically expensive because it requires step-level annotation.
Also called: PRM, process reward models
Why it matters: By rewarding good reasoning steps, not just correct endings, it gives much richer guidance during training, at the cost of expensive step-by-step labeling.
For example, instead of only checking the final answer to a math problem, it grades each line of the working to catch a mistake midway.
Heard on the show
“What you actually want is a process reward model, a PRM, that scores each step: was this move progress?”Episode 173 — The Free Step-Level Grader Hiding in Every RL Training Run