Definition
Plain language
A score, sibling to the p-value, for whether a reported finding is a typical result the data supports or was fished from the extreme edge of everything the data could have said.
As stated in the literature
A statistic measuring an analysis's fragility to analyst choice: holding the data fixed, the fraction of defensible analyses (from an Agentic Bootstrap reference distribution) whose conclusion lands at least as far from the crowd as the reported one; complements the p-value's fragility-to-noise.
Also called: m-values
Why it matters: It gives readers a way to judge how fragile a reported result is to the analyst's choices, complementing the familiar p-value's focus on random noise.
For example, an m-value tells you whether a headline finding is what most reasonable analyses of the data would have found, or a rare result cherry-picked from the far edge of what's possible.
Heard on the show
“The formal core of the paper is here — the m-value — and it pays off as a single number that says whether any reported claim, human or AI, was fished from the edge of this spectrum.”Episode 196 — AI Agents Reached Opposite Conclusions From the Same Data — and Passed Review