Definition
Plain language
The starting point you compare everything else against.
As stated in the literature
A reference condition, model, or prompt used as the comparison arm for measuring the effect of an intervention.
Also called: baselines
Why it matters: Without it, an impressive-looking number means nothing, because you can't tell whether the change helped, hurt, or did nothing at all.
For example, before claiming a new prompt helps, you first record how the plain unmodified prompt scores so you have something to compare against.
Heard on the show
“Two reasonable baselines fail on different axes.”Episode 002 — An AI Ran a Real Optics Lab for 21 Hours and Found a Transformer-Shaped Pattern in Light