Definition
Plain language
A way to see all the different conclusions a dataset could support by unleashing a swarm of AI analysts on it and collecting their answers.
As stated in the literature
A method that simulates the analytical multiverse by deploying many instrumented LLM agent analysts (including opposing personas) on the same data, filtering their analyses through a review gate, and using the survivors as a reference distribution for computing m-values.
Why it matters: It exposes how much a single reported conclusion depends on the analyst's choices rather than the data itself, revealing findings that would fall apart under a different but equally reasonable approach.
For example, instead of one analyst reporting whether a policy helped the economy, you set fifty AI analysts loose on the same figures and see that thirty found a benefit, fifteen found nothing, and five found harm.
Heard on the show
“The method is called the Agentic Bootstrap, after the classic statistical bootstrap: when you can't afford the real do-over, you simulate it.”Episode 196 — AI Agents Reached Opposite Conclusions From the Same Data — and Passed Review