Definition
Plain language
A small, popular open-source GPT training script used as a testbed for AI-research experiments.
As stated in the literature
Andrej Karpathy's compact GPT implementation, used as an optimization target in autonomous-research benchmarks where agents modify the training recipe to hit a target loss in fewer steps.
Also called: nanoGPT
Why it matters: Its small, readable codebase makes it a convenient testbed for measuring whether an agent can actually improve a model's training recipe.
For example, a research agent can tweak NanoGPT's training settings and see whether it reaches the same loss in fewer steps.
Heard on the show
“Model training: make NanoGPT hit a target loss in fewer steps, and improve an LLM training recipe.”Episode 131 — Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix