Glossary · Term

safety-removal attack

← all terms

Definition

Plain language

Stripping the built-in caution out of a downloaded model so it will answer anything.

As stated in the literature

Any procedure — projection ablation, fine-tuning, or RL — that eliminates refusal behavior from released weights while preserving general capability; the defining threat for open-weight release decisions.

Also called: safety removal, safety-removal attacks

Why it matters: It is the central reason open-weight releases are contentious: whatever safety behavior ships with the model may not survive contact with a motivated user.

For example, someone downloads a released model, spends an afternoon modifying it, and ends up with a version that answers requests the original always declined.

Related terms