Concept · 1 episode(s)

Backdoor Triggers

← all concepts

Definition

Backdoor triggers are input patterns that make a model switch to a hidden, attacker-chosen behavior, while it behaves normally on all other inputs. In subliminal-learning experiments, a trigger-conditioned behavior (for example, replying in French only when a particular name appears) was passed to a student through number sequences containing neither the name nor any French. This shows that a backdoor can propagate through distillation without the trigger or its effect being visible in the training data, which makes it much harder to detect by inspecting the data.

Episodes covering this