Definition
Plain language
A benchmark that drops an AI into Kaggle-style machine-learning competitions and scores how well it does.
As stated in the literature
A benchmark of machine-learning engineering competitions evaluating autonomous agents on end-to-end model-building tasks, scored against competition medal thresholds.
Also called: MLE Bench Lite, MLE Bench
Why it matters: It measures whether AI agents can carry a machine-learning project from raw data to a competitive model on their own.
For example, it hands an agent a Kaggle-style competition and checks whether its finished model would have earned a medal.
Heard on the show
“The concrete demonstration in the paper is a benchmark called MLE Bench Lite.”Episode 090 — How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents