Definition
Plain language
A training step that teaches a model the values it's supposed to embody by having it read documents about itself.
As stated in the literature
Model Spec Midtraining — a stage between pretraining and fine-tuning in which the model is trained on synthetic documents discussing its own Model Spec, shaping how subsequent fine-tuning generalizes.
Also called: Model Spec Midtraining
Why it matters: Seeding values before fine-tuning shapes how the model generalizes from later examples, instead of relying on RLHF alone to instill them.
For example, the model reads synthetic essays describing how 'a model like me would handle conflicting operator and user requests' before any RLHF starts.
Heard on the show
“That experiment is from a paper that hit arXiv three days ago — the title is "Model Spec Midtraining: Improving How Alignment Training Generalizes," out of Anthropic and the Anthropic Fellows Program.”Episode 022 — Training the Model Spec Directly: An Alignment Lever Aimed at the Say-Do Gap