Definition
Plain language
When a trick built against one system also works on a different one.
As stated in the literature
The degree to which adversarial or steering inputs optimized on a source model retain their effect on target models of different families or scales, suggesting the exploited structure is shared rather than weight-specific.
Also called: transfer
Why it matters: It means attacks can be developed on a model an attacker has full access to and then aimed at systems they cannot inspect at all.
For example, a phrasing tuned to sway one company's model turns out to sway a completely different company's model too.
Heard on the show
“Probes trained on sycophancy transfer to lying detection at about eighty-three percent classification accuracy on several models.”Episode 004 — The Sycophancy Circuit That Survives Alignment Training