Definition
Plain language
The problem of supervising an AI that may be smarter than its supervisors.
As stated in the literature
The challenge of constructing reliable supervision for capable AI systems using weaker or cheaper overseers, including debate, prover-verifier games, and conservative-agency approaches.
Why it matters: As AI grows more capable than its supervisors, finding ways to keep it honest and on track becomes essential to deploying it safely.
For example, it asks how a person could reliably check the work of an AI that reasons faster and deeper than they can.
Heard on the show
“The full annotated version is up on paperdive dot AI — every term tap-to-define, with links to the related work on scalable oversight and reward hacking, grouped by theme.”Episode 207 — An AI Graded Its Own Math Test 94 Percent — It Actually Scored 20