Definition
Plain language
A safety approach that assumes an AI might be untrustworthy and wraps procedures around it to stay safe anyway, instead of just hoping the AI is well-behaved.
As stated in the literature
A line of safety research focused on deploying possibly-misaligned models safely by monitoring, editing, resampling, and splicing their outputs; depends on an information asymmetry where overseers know which content is real and the model does not.
Why it matters: It offers a way to deploy powerful models safely even when you cannot fully trust that they are well-behaved.
For example, even if an AI assistant might be trying to slip in harmful code, an overseer can resample or rewrite its suspicious outputs before they ever run.
Heard on the show
“There's a whole line of safety work called AI control — the idea that even if you can't fully trust a model, you can wrap procedures around it to deploy it safely.”Episode 143 — When a Model Notices You Forged Its Own Words, And Why That Breaks Safety Tests