Definition
The superposition hypothesis is the interpretability conjecture that neural networks represent far more concepts than they have neurons by encoding each concept as a direction in activation space rather than dedicating a single neuron to it. Because these directions overlap, individual neurons look polysemantic — firing for seemingly unrelated inputs — and recovering the underlying features, e.g. with a sparse autoencoder, is what makes the model’s internals legible.