Definition
Plain language
The portion of a large model that actually gets switched on for each word it processes.
As stated in the literature
In a mixture-of-experts model, the subset of weights routed to per token (e.g. ~3B active out of 30B total), which determines compute cost per token while total parameters determine capacity and memory footprint.
Also called: active parameter
Why it matters: It separates how much a model knows from how much it costs to run, letting builders grow capacity without paying full compute price on every token.
For example, a model advertised as having 30 billion parameters might only switch on about 3 billion of them for each word it writes, so it runs closer to the speed of a much smaller model.