Definition
Plain language
An extra training stage slotted between the big initial training and the final fine-tuning, used to shape what a model believes about itself before it's polished into an assistant.
As stated in the literature
A training phase between pretraining and supervised fine-tuning; in Model Spec Midtraining, the model is trained on synthetic documents describing its own intended character so that subsequent fine-tuning generalizes differently.
Also called: mid-training
Why it matters: It matters because shaping a model's self-understanding early can change how well its final behavior generalizes.
For example, before final polishing, a model might be trained on documents describing the kind of assistant it's meant to be, so that later fine-tuning takes hold more consistently.
Heard on the show
“Stage one is what they call mid-training.”Episode 082 — Training a Deep Research Agent on 8,000 Synthetic Tasks: The Rubric Tree Trick