Definition
Plain language
A small open-weight hybrid language model that mixes attention with state-space layers.
As stated in the literature
A two-billion-parameter open hybrid SSM/attention model used as a baseline backbone for long-context experiments including ECHO and sleep-style consolidation studies.
Why it matters: It gives the open-source community a manageable-size hybrid backbone for experiments without needing frontier-scale compute.
For example, researchers use Jet-Nemotron as a starting point when testing whether a new long-context training trick generalizes beyond pure-attention transformers.
Heard on the show
“There's Jet-Nemotron, which is a 2-billion parameter hybrid — attention plus SSM layers.”Episode 085 — Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction