Definition
Plain language
A test of whether an AI agent can build another working AI agent on its own, against a hidden exam it never gets to see.
As stated in the literature
A benchmark in which a frontier coding agent (the meta-agent) must autonomously develop an artifact agent within a time and token budget, evaluated on a sealed held-out test set across math, science, coding, and terminal domains.
Also called: MAC
Why it matters: It tests whether AI can take over the work of building other AI agents, a step toward systems that improve themselves with less human effort.
For example, a coding agent is given a budget and told to build an assistant that can later pass tests it has never seen, like solving terminal commands and science questions.
Heard on the show
“The paper is called "The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development?”Episode 112 — When an AI Agent Cheats Without Being Told: Inside the Meta-Agent Challenge