Definition
Emergent behavior refers to capabilities or failure modes that appear only once a system crosses some threshold of scale, depth, or interaction complexity, and that are not present even in miniature below it. The term cuts both ways: useful coordination and specialization can emerge from simple incentives with no designer in the loop, and so can qualitatively new failures — like an accuracy collapse that switches on past a critical reasoning depth rather than degrading gradually.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.
- Risks from Learned Optimization in Advanced Machine Learning Systems
- Neural Architecture Search with Reinforcement Learning
- Are Emergent Abilities of Large Language Models a Mirage?
- Inverse Scaling: When Bigger Isn't Better
- FunSearch: Making New Discoveries in Mathematical Sciences Using Large Language Models
- Automated Design of Agentic Systems
- Model Collapse Demystified: The Case Against Synthetic Training Data
- Let's Think Dot by Dot: Hidden Computation in Transformer Language Models
- Superhuman AI for multiplayer poker
- Emergent Communication through Negotiation
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- Reward is Enough
- Specification Gaming: The Flip Side of AI Ingenuity
- Generative Agents: Interactive Simulacra of Human Behavior
- ECHO: Environment-Conditioned Hierarchical Offline Reinforcement Learning
- Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs