Definition
Plain language
Harm that only appears once an AI does the same small, innocent-looking action thousands of times.
As stated in the literature
A misuse category from phone-agent safety work where each individual action is indistinguishable from legitimate use, and the violation exists only in the aggregate volume or the intent behind it; contrasted with overt misuse, whose harmful payload is visible in a single step.
Why it matters: It highlights harms that no single-step safety check can catch, because the danger lives in volume or intent rather than any one action.
For example, an AI that posts one harmless-looking comment is fine, but the same AI posting thousands of them to flood a forum becomes a problem only in the aggregate.
Heard on the show
“And that connects to a concept the paper introduces that's worth slowing down for — emergent misuse.”Episode 185 — Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway