Definition
Plain language
A memory-saving tweak to attention where several of the model's attention units share the same lookup keys.
As stated in the literature
An attention variant in which multiple query heads share a smaller set of key-value heads, reducing KV-cache size and inference memory with minimal quality loss; standard in many recent transformers.
Also called: GQA
Why it matters: It shrinks the memory cost of running large models with little quality loss, helping them serve more users faster and cheaper.
For example, instead of every attention unit keeping its own lookup table, several units share one table, cutting the memory the model needs while running.
Heard on the show
“Sixty-two transformer layers, grouped-query attention, a standard speculative-decoding side module.”Episode 090 — How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents