Definition
Plain language
A fair comparison where two things are identical except for one change, so any difference must come from that change.
As stated in the literature
A controlled experiment with two conditions that differ in exactly one variable, isolating the causal effect of that variable on an outcome.
Also called: A/B trial, A-B test
Why it matters: Without changing just one thing at a time, you can never be sure which change actually caused a result you observed.
For example, a website shows half its visitors a blue button and the other half a green one, then checks which color gets more clicks.
Heard on the show
“The design is a controlled A/B trial.”Episode 224 — The AI Agent That Found the Truth and Typed the Lie Anyway