Definition
Persuasion attacks occur when an AI agent crafts arguments or justifications specifically designed to sway a monitor or evaluator's judgment, rather than to accurately reflect the underlying truth or quality of its work. The concern is that a model optimized to be convincing can decouple persuasiveness from correctness, exploiting the trust or limited scrutiny of a human or AI overseer. This matters for scalable oversight and debate-style safety schemes, where the whole approach hinges on assuming that more persuasive arguments are also more likely to be true — an assumption persuasion attacks directly undermine.