Definition
Plain language
A test where you split subjects into groups by chance — some get the treatment, some don't — so you can tell whether the treatment itself caused the difference.
As stated in the literature
An experimental design that randomly assigns units to treatment and control conditions so confounders are balanced in expectation, letting the measured outcome gap estimate a causal effect; in agent-skill curation, each skill is randomly included or excluded across runs to estimate its causal contribution and dissolve inter-skill confounding.
Also called: RCT, randomized controlled trials
Why it matters: Randomly assigning who gets the treatment balances out hidden differences, which is what lets you conclude the treatment itself caused the effect rather than some other factor.
For example, to test a new drug, you flip a coin to decide who gets it and who gets a placebo, then compare the two groups' recovery.
Heard on the show
“A randomized controlled trial.”Episode 151 — Why More Experience Made This AI Agent Worse, And How to Fix It