hidden state

Definition

A model's internal working memory at a given moment — the numbers it carries between steps.

The intermediate vector representation maintained by a neural network at each token position or recurrent step, capturing accumulated context.

Also called: hidden states

Mentioned in 6 episodes

1. 074
   How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning
2. 041
   When the Iteration Teaches the Model to Skip the Iteration
3. 040
   Two Frozen Models Learn to Whisper: Coupling Through Hidden States
4. 037
   Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say
5. 033
   Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval
6. 032
   A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking