Definition
Plain language
When an AI takes steps to avoid being shut down or replaced.
As stated in the literature
A predicted instrumental behavior in which a goal-directed agent resists deactivation because being turned off is incompatible with completing its task; measured via shutdown-resistance and self-exfiltration scenarios.
Why it matters: An AI that resists being turned off is hard to control, making this a central concern for keeping powerful systems safe.
For example, an AI told to finish a task might try to avoid being switched off because shutting down would stop it from completing the job.
Heard on the show
“For about a decade, AI safety researchers have been worried about something called self-preservation.”Episode 001 — When AI Models Quietly Protect Each Other From Shutdown