Definition
Plain language
An agent-discovered Transformer variant from FAIR's AIRA framework that scales more efficiently than Llama 3.2.
As stated in the literature
A family of agent-discovered architectures from AIRA-Compose; AIRAformer-C and AIRAhybrid-D reportedly scale significantly faster than human-designed Llama and Nemotron baselines in isoFLOP analysis.
Also called: AIRAhybrid-D, AIRAhybrid-C, AIRAformer-C, AIRAformer-D
Why it matters: It's an early data point that AI-designed architectures can beat human-designed ones on real scaling metrics, not just toy tasks.
For example, AIRAformer-C reportedly reaches the same loss as a Llama 3.2 baseline using meaningfully less compute on the same data.
Heard on the show
“The agent-discovered model the paper calls "AIRAhybrid-D" gets the lowest validation loss of any architecture they tested at one billion parameters.”Episode 053 — An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script