Definition
Plain language
How well someone can predict what a model will do just from the model's own description of itself.
As stated in the literature
An interpretability and transparency metric measuring how accurately an external observer can predict a model's behavior given its stated rules or explanation; in Self-CTRL, a third-party auditor predicting refuse-versus-comply purely from the model's self-described policy.
Why it matters: A model's self-explanation is only useful if it lets people accurately anticipate its behavior, which is what this measures.
For example, if a model says it refuses medical advice but answers health questions anyway, an observer trying to predict it from its stated rule will guess wrong.
Heard on the show
“At the transparency end, the auditor's ability to predict behavior shoots way up — that's the big simulatability win — but safety only nudges.”Episode 152 — Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good