Definition
Plain language
NVIDIA's family of hybrid foundation models that mix attention with state-space layers.
As stated in the literature
NVIDIA's hybrid SSM/Transformer model family used as a comparison baseline for agent-discovered architectures in AIRA-Compose isoFLOP analysis.
Why it matters: It serves as a real-world hybrid baseline to compare against when claiming new architectures beat the best existing designs.
For example, a Nemotron-H model interleaves Mamba-style state-space layers with standard attention layers to reduce inference cost on long contexts.
Heard on the show
“Models like Jamba, Nemotron-H, Olmo-Hybrid.”Episode 027 — When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure