Definition
Plain language
The little range around a reported number showing how much it might wobble if you measured again.
As stated in the literature
A visual or numeric representation of estimate uncertainty (standard deviation, standard error, or confidence interval); single-seed results carry none, so differences of a few points may be path dependence rather than signal.
Also called: error bar
Why it matters: Without them, small gaps between systems look like real differences when they may just be luck, which quietly corrupts every leaderboard built on them.
For example, reporting a score as "72, give or take 5" tells you that a rival scoring 74 might not actually be better.
Heard on the show
“" I think the gap — fifty-seven versus a hundred and ninety-four — is big enough to dwarf the error bars and mean something real.”Episode 100 — How a Prompt Wrapper Lets a Frontier Model Play Poker Like an Expert