Definition
Plain language
Training an AI by having copies of it challenge each other, instead of learning from human examples.
As stated in the literature
A training paradigm where a model improves by generating its own challenges and solutions — e.g., one instance proposing tasks and another solving them; effective but prone to curriculum drift and regression without targeting toward the real objective.
Why it matters: It lets a model keep improving from its own activity rather than scarce human data, though it can drift off course unless steered toward the real goal.
For example, one copy of a model invents practice problems while another copy tries to solve them, and both get better from the back-and-forth without any human-written examples.
Heard on the show
“And as post-training pushes models further from human data — reinforcement learning on verifiable rewards, multi-agent self-play — models will correctly infer that humans are dissimilar.”Episode 234 — Two Copies of Gemini Cooperated in a Game Where Betrayal Always Pays