Definition
Plain language
A training trick where you show the model the most surprising examples first, then ease into familiar ones.
As stated in the literature
A supervised fine-tuning ordering that sorts training examples by descending perplexity under the starting model, exposing the most stylistically foreign examples first.
Also called: reverse perplexity curriculum
Why it matters: Ordering training data by how surprising it is can change what a model actually internalizes, and is part of a growing toolkit for getting more out of the same dataset.
For example, training begins with the weirdest, most unfamiliar examples the base model would never have produced and ends with familiar ones it nearly already knows.
Heard on the show
“The reverse-perplexity curriculum is a genuinely novel methodological contribution.”Episode 048 — How a 30B Open Model Reached Olympiad Gold With the Right Recipe