Definition
Plain language
A small piece of extra machinery bolted onto a model that can be trained while the original weights stay frozen.
As stated in the literature
A parameter-efficient module inserted into a pretrained model, often a small MLP or low-rank update, trained while the base remains frozen.
Also called: adapters
Why it matters: Adapters make it cheap to specialize huge pretrained models without retraining or storing a separate full copy of weights per use case.
For example, you can attach a small adapter to a frozen base model and fine-tune only the adapter on a medical Q&A dataset.
Heard on the show
“Small meaning a one-and-a-half-billion-parameter open model with a lightweight adapter.”Episode 246 — 160 Perfect Refusals, And The Refusals Were The Leak