Definition
Plain language
When the way you pick cases skews what you find, even if each case is studied honestly.
As stated in the literature
A statistical pattern where the sample selection process correlates with the outcome variable, biasing estimates; flagged for instance in QUEST's auto-formalization filtering and SIA's verifier-friendly benchmark choices.
Why it matters: It's the silent enemy of benchmark-driven research: the very filter that lets you build a dataset cheaply can guarantee the data isn't representative.
For example, if you only train an AI on math problems whose answers your auto-grader can verify, the resulting model gets very good at easily-checkable math — and possibly worse elsewhere.
Heard on the show
“The first thing — and the authors are upfront about this — is the selection bias on the Erdős problem set.”Episode 067 — An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won