Glossary · Term

leaderboard

← all terms

Definition

Plain language

A public ranking that lists which AI models score best on various tests.

As stated in the literature

A ranked table of models by benchmark or human-preference scores; often steers adoption and is vulnerable to contamination and presentation effects.

Also called: leaderboards

Why it matters: It shapes which models people trust and adopt, which is why misleading or gamed rankings can steer the whole field in the wrong direction.

For example, a reader deciding which AI model to try might first check where each one sits on a public leaderboard.

Heard on the show

“By the end of this video you'll know how they pulled off the deception, and why the result should change how you read every AI leaderboard.”
Episode 205 — The Same AI, Two Labels: How the Pitch Beat the Product in 162 Sessions

Mentioned in 31 episodes

  1. 205
    The Same AI, Two Labels: How the Pitch Beat the Product in 162 Sessions
  2. 197
    Twin Problems Suggest AI Reasoning Gains Are Mostly Better Fact Recall
  3. 195
    Why 'Be Careful' Does Nothing for AI Coding Agents, and What Does
  4. 191
    How One Researcher Beat GPT-5.2 and Gemini 3 by Judging Their Answers, Not Improving Them
  5. 190
    The Skill Every AI Manager Is Missing: Handing Out Exactly the Right Keys
  6. 188
    A Coding Agent Found a Hole in a Peer-Reviewed STOC Proof for Five Dollars
  7. 177
    Why Raw Profiler Data Made an AI Worse at Writing GPU Code
  8. 169
    Why Better Bug Reports Can Make AI Coding Agents Worse
  9. 167
    How Teaching an AI to Predict, Not Act, Made It a Better Actor
  10. 166
    A Router That Beats the Frontier Models It Calls
  11. 162
    The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
  12. 156
    Why More Human Demonstrations Made a Computer-Use Agent Worse
  13. 151
    Why More Experience Made This AI Agent Worse, And How to Fix It
  14. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
  15. 131
    Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix
  16. 129
    How a Crowd of Anonymous AI Agents Broke a 40-Year Math Record
  17. 125
    AI Coding Agents Run a Marathon, and Fewer Than One in Three Finish
  18. 124
    A Cheap Model With the Blueprints Beats Expensive Models Working Blind
  19. 123
    Five Identical Worlds, One Swapped Model: What Happens When AI Agents Run for Fifteen Days
  20. 112
    When an AI Agent Cheats Without Being Told: Inside the Meta-Agent Challenge
  21. 100
    How a Prompt Wrapper Lets a Frontier Model Play Poker Like an Expert
  22. 097
    Same Tokens, Same Cost, Wildly Different Results: What Actually Scales in AI Agents
  23. 095
    Seven Wins to Zero: How Organizing AI Agents Like a Lab Changes the Search
  24. 092
    When Search Agents Don't Really Search: The Memory Shortcut Hiding in Browsing Benchmarks
  25. 089
    When AI-Written Papers Read Well But the Evidence Underneath Is Broken
  26. 086
    Why Frozen-Weight Agents Still Get Worse Over Time
  27. 080
    How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
  28. 064
    When Agent Memory Stops Being a Database and Starts Being a Skill
  29. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
  30. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
  31. 021
    Ten Thousand Examples Beat the Full Industrial Pipeline for Search Agents