Definition
Plain language
NVIDIA's family of hybrid foundation models that mix attention with state-space layers.
As stated in the literature
NVIDIA's hybrid SSM/Transformer model family used as a comparison baseline for agent-discovered architectures in AIRA-Compose isoFLOP analysis.
Why it matters: It serves as a real-world hybrid baseline to compare against when claiming new architectures beat the best existing designs.
For example, a Nemotron-H model interleaves Mamba-style state-space layers with standard attention layers to reduce inference cost on long contexts.
Heard on the show
“What keeps this from being a pure benchmark artifact, though, is that a subset of this data went into the training mix for one of NVIDIA's actually-shipped models — Nemotron 3 Nano Omni.”Episode 156 — Why More Human Demonstrations Made a Computer-Use Agent Worse