Definition
Plain language
Models or tasks where the input can be tens or hundreds of thousands of words long.
As stated in the literature
The regime where transformer input sequences are large enough that attention cost, memory pressure, and degradation of recall become primary engineering and modeling concerns.
Why it matters: It's the setting where most engineering tradeoffs in modern LLM serving and architecture design currently play out.
For example, asking a model to answer questions about a 200-page contract in a single prompt puts it firmly in the long-context regime.
Heard on the show
“Yeah, the input-length regimes — they have four of them — are sampled from a pool of benchmarks specifically to construct progressively heavier long-context loads.”Episode 016 — Why Your Coding Agent Stalls While the GPU Runs Hot