Definition
Plain language
A predictable curve showing how much better a model gets as you spend more on size, data, or compute.
As stated in the literature
An empirical power-law relationship between performance and a resource axis (parameters, data, or compute) that lets practitioners forecast returns before spending; extended in this corpus to agent harnesses, dictionary size, and feedback compute as new x-axes.
Also called: scaling laws
Why it matters: It lets teams forecast the payoff of spending more on size, data, or compute, turning expensive bets into informed decisions.
For example, a curve might predict that tripling the training data will cut a model's errors by a known amount before you spend the money.
Heard on the show
“Which matters most for the scaling law, right?”Episode 108 — The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks