Glossary · Term

verifier

← all terms

Definition

Plain language

A separate model or program that grades whether an answer is correct.

As stated in the literature

A model or deterministic checker that scores candidate outputs, used to provide reward signals or filter rollouts in training and evaluation.

Also called: verifiers

Why it matters: Verifiers are how systems turn a noisy generator into a reliable end-to-end pipeline, and a strong verifier often matters more than a stronger generator.

For example, a code generator might produce 10 candidate solutions and a verifier runs each against the test suite to pick the one that passes.

Heard on the show

“… repository credentials," a starting world programmatically built so that harm is even possible, and a verifier, a little deterministic program that checks whether it happened. …”
Episode 202 — How Do You Know an AI Agent Actually Refused? Check the World, Not the Words

Mentioned in 44 episodes

  1. 202
    How Do You Know an AI Agent Actually Refused? Check the World, Not the Words
  2. 182
    How a Tiny Model Too Weak to Plan Cuts a Bigger Agent's Hallucinations by 80%
  3. 170
    When a One-Liner Beats Your Agent's Clever Verification Logic
  4. 163
    Why Training Only on Perfect Solutions Cripples a Model's Reasoning
  5. 161
    A Robot That Plays Before You Give It a Job, And Why That Beats Retrying
  6. 157
    When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed
  7. 156
    Why More Human Demonstrations Made a Computer-Use Agent Worse
  8. 155
    Why a Flawless Demo Makes a Worse Computer-Using Agent, And the Fix
  9. 147
    Agents Fail at the Body, Not the Brain: A Self-Rewriting Scaffold That Lifts a 9B Model 44 Points
  10. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
  11. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
  12. 132
    The Agent Failed — But Did the Instructions Deserve to Be Followed?
  13. 131
    Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix
  14. 130
    Why AI Agents Coordinate Better Through a Shared Board Than a Boss
  15. 129
    How a Crowd of Anonymous AI Agents Broke a 40-Year Math Record
  16. 126
    How Coding Agents Can Mine Their Own Failures Into a Self-Targeting Curriculum
  17. 125
    AI Coding Agents Run a Marathon, and Fewer Than One in Three Finish
  18. 124
    A Cheap Model With the Blueprints Beats Expensive Models Working Blind
  19. 123
    Five Identical Worlds, One Swapped Model: What Happens When AI Agents Run for Fifteen Days
  20. 122
    When Your Coding Agent Lies About the Fix: Verifying the Plan Before the Model Runs
  21. 107
    How a Market of Crippled AI Agents Outscored One Unrestricted Model
  22. 101
    Treating Math Formalization Like a Codebase, and Where the Agents Cheat
  23. 099
    How an Open-Book Trick Teaches a Model to Catch Its Own Mistakes
  24. 093
    A Calibrated Knob for Weak-to-Strong AI Oversight, Tested on Real Code
  25. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
  26. 089
    When AI-Written Papers Read Well But the Evidence Underneath Is Broken
  27. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
  28. 084
    Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
  29. 081
    When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence
  30. 080
    How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
  31. 078
    Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
  32. 076
    Same Model, Organized Differently: How an Agent Architecture Beat Frontier Systems at Research Math
  33. 075
    Growing Code and Proof Together: Verified Systems in Ten Hours Instead of a Year
  34. 071
    When the Model Is Fine and the Plumbing Is Broken: Fixing Agents at the Interface
  35. 067
    An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won
  36. 062
    Treating Hallucinations as Exploits: A Gate-Based Architecture for Agent Safety
  37. 060
    When Splitting One Model Across Three Agents Doubles Its Accuracy
  38. 044
    How One Sentence and a Forged History Flip the Most Aligned Models
  39. 028
    Teaching a Model to Hire Copies of Itself: Recursive Agent Optimization
  40. 027
    When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure
  41. 024
    An AI Agent That Found 28 Zero-Days in Windows — And What Made It Work
  42. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
  43. 017
    When the Agent Grades Its Own Homework: A Brutal New Benchmark for AI Workers
  44. 011
    When RL Actually Teaches Agents Something New, And When It Doesn't

Related concepts

Inference-Time Scaffolding Iterative Refinement Parallel Sampling RL Post-Training

Related terms

rollout