Definition
Plain language
A setting you choose before training a model, like how fast it should learn, as opposed to something the model figures out on its own.
As stated in the literature
A configuration value set prior to training (learning rate, batch size, schedule, regularization strength) rather than learned; suboptimal choices can confound baseline comparisons independent of architecture or data.
Also called: hyperparameters
Why it matters: A poorly chosen setting can make a model look worse than a rival regardless of its design, so unfair comparisons often trace back to these choices.
For example, you decide how fast the model should learn before training even begins, rather than letting it discover that pace on its own.
Heard on the show
“But they ran their SFT baseline using one of the standard frameworks, and they ran a separate SFT baseline using a different framework — same data, same model, same hyperparameters.”Episode 009 — How Two Silent Library Bugs Quietly Invalidated a Wave of Reasoning Papers