Mamba

Definition

A type of language model that maintains a fixed-size memory instead of attending to every previous token.

A state-space model architecture that processes sequences with selective recurrent state updates, offering linear-time inference and constant memory per layer.

Also called: Mamba-2, Mamba-3

Mentioned in 3 episodes

1. 053
   An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script
2. 033
   Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval
3. 032
   A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking

Related concepts