long-context

Definition

Models or tasks where the input can be tens or hundreds of thousands of words long.

The regime where transformer input sequences are large enough that attention cost, memory pressure, and degradation of recall become primary engineering and modeling concerns.

Mentioned in 7 episodes

1. 090
   How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
2. 085
   Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction
3. 079
   An Old Idea From Cognitive Psychology Reshapes How We Reward Reasoning Models
4. 036
   Sparse Attention Was the Wrong Frame. Treat It as Geometry Instead.
5. 033
   Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval
6. 028
   Teaching a Model to Hire Copies of Itself: Recursive Agent Optimization
7. 016
   Why Your Coding Agent Stalls While the GPU Runs Hot

Related concepts