Definition
Plain language
A natural-language checklist describing what a good answer to a question looks like.
As stated in the literature
A structured natural-language criterion list used in rubrics-as-rewards training to provide process-level supervision signal; expensive due to per-problem authoring requirements.
Also called: rubrics-as-rewards, rubrics
Why it matters: Rubrics give RL training a denser, more interpretable signal than a single right/wrong reward, but they're expensive because each problem needs its own thoughtfully-written list.
For example, the rubric for a strong essay might list "introduces the topic clearly," "cites at least two sources," and "explicitly addresses the counterargument."
Heard on the show
“Now imagine you're writing a rubric to grade student answers.”Episode 019 — When the Best Reward Model Trains the Worst Policy: Inside EvoLM