Definition
Plain language
A range of values that probably contains the true number you're trying to measure.
As stated in the literature
An interval estimate around a measured statistic expressing the uncertainty of the estimate; non-overlapping intervals are used as informal evidence that two measured groups genuinely differ.
Also called: confidence intervals
Why it matters: It tells you how much to trust a measured number and offers a quick way to judge whether two results really differ or just look different by chance.
For example, a poll might report that 52% of people favor a policy, give or take 3 percentage points, meaning the true figure likely sits between 49% and 55%.
Heard on the show
“The bootstrap confidence interval on the capability-expansion statistic is something like two to eight problems — well above zero, but a wide band around the five-to-one point estimate.”Episode 011 — When RL Actually Teaches Agents Something New, And When It Doesn't