Definition
Plain language
A shortcut wire in a neural network that adds the input of a layer back to its output.
As stated in the literature
An additive skip connection that adds a block's input to its output, originally introduced in ResNet, foundational in transformer architectures.
Also called: skip connection
Why it matters: Residual connections are what made very deep networks trainable in the first place, and modern transformers depend on them to pass information across dozens of layers.
For example, in a deep network a layer might output `f(x) + x` instead of just `f(x)`, so the original input is preserved alongside the layer's transformation.