Definition
Plain language
A bank of switchable internal units in a model where each data source gets its own private slice, so one source can be turned off without disturbing the rest.
As stated in the literature
In natively-unlearnable architectures, feed-forward neurons gated by a per-source pseudo-random mask, so each source's unique knowledge migrates into its own switchable subset separate from an always-on shared backbone; flipping a source's mask cleanly removes its unique content.
Also called: sink neuron
Why it matters: It makes it possible to cleanly remove one data source's contribution from a trained model, useful when content must be deleted or unlearned.
For example, each website a model learned from gets its own dedicated set of internal switches, so that source's unique knowledge can be flipped off later without disturbing what the model learned elsewhere.
Heard on the show
“And then a huge bank of switchable bulbs — the sink neurons — where any given source is only allowed to light up a tiny, specific handful.”Episode 145 — Building Forgetting Into a Language Model With One Extra Line of Code