Definition
Plain language
A hybrid open-weight language model that mixes attention layers with state-space layers.
As stated in the literature
AI21 Labs' hybrid SSM/Transformer foundation model, demonstrating the production move away from pure-attention architectures.
Why it matters: It's an early production demonstration that hybrid architectures can deliver competitive quality with much better long-context efficiency.
For example, Jamba interleaves Transformer attention blocks with Mamba state-space blocks so it can handle long inputs without paying full attention cost on every layer.
Heard on the show
“Models like Jamba, Nemotron-H, Olmo-Hybrid.”Episode 027 — When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure