Definition
Plain language
Comparing two related AI models to see exactly what changed between them — which internal features one has that the other doesn't.
As stated in the literature
The interpretability task of identifying representational differences between a base model and a fine-tuned (e.g., RL-trained) version, often via a crosscoder that separates shared from model-exclusive features; aims to localize what training installed.
Why it matters: It lets researchers pinpoint exactly what a round of training changed inside a model, instead of only noticing differences in its behavior.
For example, comparing a chatbot before and after a training round can reveal the specific internal feature that now makes it refuse certain requests.
Heard on the show
“The reason you'd want that is what the field calls model diffing — you get to see which features both models use, versus which ones belong to only one.”Episode 175 — One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent