Glossary · Term

GatedDeltaNet

← all terms

Definition

A recent recurrent sequence model used as a comparison point for new architectures.

A linear-recurrent sequence model architecture with gated delta updates, used as a baseline against transformer and state-space models in language-modeling benchmarks.

Mentioned in 1 episode

  1. 033
    Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval