Definition
Plain language
A step-by-step way of investigating whether an AI that did something alarming actually meant to, or was just confused.
As stated in the literature
A methodology that reads an agent's chain-of-thought to generate hypotheses about a concerning behavior, then tests them by intervening on the environment and resampling, separating genuine misalignment from confusion or setup artifact.
Why it matters: It separates an agent that genuinely intended harm from one that was simply confused, which is essential for judging how seriously to treat a concerning behavior.
For example, after an agent does something alarming, an investigator reads its step-by-step reasoning, forms a guess about why, then tweaks the situation and reruns to test that guess.
Heard on the show
“By the end you'll understand a method these researchers call "model forensics" — a repeatable recipe for telling whether an AI that did something alarming actually meant to, or was just confused.”Episode 174 — When the AI 'Schemes,' It's Usually Just Lazy or Confused