Definition
Plain language
Running an analysis every reasonable way at once and reporting the whole spread of answers, instead of just the one path you happened to take.
As stated in the literature
A methodology that enumerates and executes many defensible analytical specifications of the same data to characterize the full distribution of possible conclusions; historically labor-intensive and rare, made cheap by agentic approaches like the Agentic Bootstrap.
Why it matters: It reveals how much a conclusion depends on analytical choices instead of the data, guarding against results that only hold under one particular path.
For example, rather than reporting the one result from the way they analyzed the data, researchers run all the reasonable variations and show the full range of answers those choices produce.
Heard on the show
“People called it multiverse analysis and almost nobody did it.”Episode 196 — AI Agents Reached Opposite Conclusions From the Same Data — and Passed Review