Glossary · Term

decoding

← all terms

Definition

Plain language

The step-by-step process of a language model turning its internal numbers into actual words, one piece at a time.

As stated in the literature

The procedure converting a model's output probability distribution into emitted tokens via strategies like greedy selection, nucleus sampling, or speculative decoding; the regime where commitment failures and latency tradeoffs surface.

Also called: decode, decoded

Why it matters: The decoding strategy shapes both how good and how fast an answer is, trading off quality, variety, and the time you wait for a reply.

For example, when a chatbot replies word by word, decoding is the step choosing each next word from the model's internal list of probabilities.

Heard on the show

“These characters are correctly-decoded, valid Unicode with no assigned picture, so the renderer just silently drops them.”
Episode 208 — The Blank Space in Your AI Approval Box That Isn't Empty

Mentioned in 24 episodes

  1. 208
    The Blank Space in Your AI Approval Box That Isn't Empty
  2. 206
    How Four-Second Clips Become Hours of Playable AI Soccer
  3. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
  4. 203
    The Thought a Model Doesn't Say — and the Lens That Reads It
  5. 197
    Twin Problems Suggest AI Reasoning Gains Are Mostly Better Fact Recall
  6. 179
    How DeepSeek Made One User Faster Without Slowing Down the Crowd
  7. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
  8. 158
    How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave
  9. 149
    When Cornering a Chatbot Makes It Lie: J.P. Morgan's Case for 'Playing Dead'
  10. 146
    How an Innocent README Can Freeze an AI Agent's Safety Check for an Hour
  11. 140
    When a Reasoning Model Says "Let Me Double-Check" After It's Already Decided
  12. 116
    Why Streaming Half a Reasoning Chain Beats Sending the Whole Thing
  13. 102
    How to Catch an AI Attack That No Single Conversation Reveals
  14. 094
    Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most
  15. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
  16. 077
    Reading a Model's Confidence Curve to Decide When Chain-of-Thought Is Worth It
  17. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
  18. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
  19. 041
    When the Iteration Teaches the Model to Skip the Iteration
  20. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
  21. 036
    Sparse Attention Was the Wrong Frame. Treat It as Geometry Instead.
  22. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
  23. 027
    When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure
  24. 018
    Language Models Compute the Rational Move, Then Override It

Related concepts

Entropy Gating Residual Stream Speculative Decoding Token-Level Analysis Variational Autoencoder

Related terms

commitment failure latency nucleus sampling speculative decoding token