Definition
Plain language
One processing stage in a neural network; information passes through many of them in sequence.
As stated in the literature
A stacked block of a network (in transformers, attention plus MLP with residual connections); interpretability studies track where in the layer stack a representation is built, refined, or discarded.
Also called: layers
Why it matters: Knowing which layer holds which information tells researchers where to look, where to intervene, and where a computation actually happens inside the network.
For example, a fact about a word might not be present in the model's early layers, appear clearly in the middle ones, and then be replaced by something else by the final layers.
Heard on the show
“That work gets handed to a separate layer of support agents whose job is to come back with a clean, curated answer.”Episode 002 — An AI Ran a Real Optics Lab for 21 Hours and Found a Transformer-Shaped Pattern in Light