Definition
Plain language
A small piece of extra machinery bolted onto a model that can be trained while the original weights stay frozen.
As stated in the literature
A parameter-efficient module inserted into a pretrained model, often a small MLP or low-rank update, trained while the base remains frozen.
Also called: adapters
Why it matters: Adapters make it cheap to specialize huge pretrained models without retraining or storing a separate full copy of weights per use case.
For example, you can attach a small adapter to a frozen base model and fine-tune only the adapter on a medical Q&A dataset.
Heard on the show
“The controlled experiment is on relatively small models, with rank-sixteen LoRA adapters, on a thousand preference pairs.”Episode 004 — The Sycophancy Circuit That Survives Alignment Training