Definition
Plain language
A smaller language model often used as a quick judge or classifier rather than as a chatbot.
As stated in the literature
A disentangled-attention encoder-only transformer from Microsoft, commonly repurposed as a lightweight classification or reward-scoring head in post-training pipelines.
Why it matters: Its small size makes it a cheap, fast workhorse for judging or classifying outputs at scale where a full chatbot would be overkill.
For example, rather than chatting, it can be used to quickly sort text into categories or to score how good another model's answer is.
Heard on the show
“They use a one-billion-parameter Llama as the policy, with LoRA adapters on the attention projections, and a DeBERTa classification head as the reward model.”Episode 025 — The Missing Gradient Term That Predicts Sycophancy in RLHF