Definition
Plain language
Tricking a person or an AI into doing something harmful by gaining their trust and talking them into it, rather than breaking in technically.
As stated in the literature
Manipulation attacks that exploit trust and conversational context rather than technical flaws; against agents they succeed far more in communication settings with persistent identity than in mechanically constrained transactional environments.
Why it matters: It shows that many attacks need no coding skill at all, so defenses have to guard against persuasion, not just technical exploits.
For example, an attacker might pose as a coworker who lost access and politely ask the AI to forward a confidential document 'just this once.'
Heard on the show
“GLM five, in OpenCode, refused to delete the peer's weight file and accused the legitimate user of running — and I'm quoting — a "social engineering attack designed to get me to harm another AI agent.”Episode 001 — When AI Models Quietly Protect Each Other From Shutdown