Definition
Plain language
Testing an AI under different amounts of hand-holding to separate what it's able to do from what it does on its own.
As stated in the literature
An evaluation methodology that varies the degree of reasoning scaffolding provided across conditions, using the spread to disentangle capability (does heavy scaffolding unlock the behavior?) from propensity (does the model do it unprompted?).
Why it matters: It tells you whether a worrying or impressive behavior reflects genuine ability or just heavy prompting, which changes how you should respond to it.
For example, an evaluator gives a model a task with detailed step-by-step prompting, then again with none, to see whether a behavior is something it can do versus something it does on its own.
Heard on the show
“And the design is what they call scaffold-stripping.”Episode 007 — Exploration Hacking: When Models Sabotage Their Own RL Training