Jamba

Definition

A hybrid open-weight language model that mixes attention layers with state-space layers.

AI21 Labs' hybrid SSM/Transformer foundation model, demonstrating the production move away from pure-attention architectures.

Mentioned in 2 episodes

1. 053
   An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script
2. 027
   When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure