Definition
Plain language
Sneaking corrupted examples into the data an AI learns from, to make it behave badly later.
As stated in the literature
An attack that injects adversarial examples into a model's training corpus to implant backdoors or degrade behavior; distinguished from prompt injection and from Oracle Poisoning of runtime data sources.
Also called: training-data poisoning
Why it matters: It means a model can be sabotaged before it's even deployed, so the trustworthiness of training data directly affects safety.
For example, an attacker might slip a few thousand tampered examples into a model's training data so it misbehaves whenever it sees a secret trigger phrase.
Heard on the show
“… crowded field of named attacks on AI systems and a listener could easily mishear this as "oh, more data poisoning. …”Episode 039 — When Smarter Agents Get Fooled by Three Extra Nodes in a Database