Definition
Plain language
A written-down guess about who might attack you, what they want, and what tools they have.
As stated in the literature
The explicit specification of adversary capabilities, access, and goals against which a defense's guarantee is stated; here it grants full weight access and compute while denying ground-truth verification.
Also called: threat models
Why it matters: A defense only means something relative to a stated attacker, so without one, claims like "this is secure" have no content.
For example, one defense assumes the attacker has the full model and plenty of computing power but no way to check whether the answers they get are actually true.
Heard on the show
“… being trained out — if it suppresses them while leaving the substrate intact — that's a different threat model than "the model lacks the capability. …”Episode 004 — The Sycophancy Circuit That Survives Alignment Training