Glossary · Term

decoding

← all terms

Definition

Plain language

The step-by-step process of a language model turning its internal numbers into actual words, one piece at a time.

As stated in the literature

The procedure converting a model's output probability distribution into emitted tokens via strategies like greedy selection, nucleus sampling, or speculative decoding; the regime where commitment failures and latency tradeoffs surface.

Also called: decode, decoded

Why it matters: The decoding strategy shapes both how good and how fast an answer is, trading off quality, variety, and the time you wait for a reply.

For example, when a chatbot replies word by word, decoding is the step choosing each next word from the model's internal list of probabilities.

Heard on the show

“So back to that opening claim, which you can now fully decode.”
Episode 242 — Making a Vision Model Better by Showing It Blurry Images

Mentioned in 34 episodes

  1. 242
    Making a Vision Model Better by Showing It Blurry Images
  2. 241
    Swapping the Name Did Nothing, But Hedging Moved Every Model
  3. 238
    How a Cheap Model Reads the Flagship's Secret Reasoning Aloud
  4. 237
    The Model Built a Perfect Map of the Puzzle, Then Lost It
  5. 232
    Coding Models Can Find the Bad Line, They Just Won't Delete It
  6. 230
    Why AI Survey Panels Break Before the Dice Ever Roll
  7. 227
    Poisoned Bug Reports Fooled Coding Agents Two Times Out of Three
  8. 225
    How a Frozen Model Went From Zero to Sixty Percent by Borrowing Another's Thinking
  9. 219
    Forty-Four AI Models, One Word, And The Newest Ones Conform Most
  10. 215
    The Same Policy Scored 85 for the US and 36 for Russia
  11. 208
    The Blank Space in Your AI Approval Box That Isn't Empty
  12. 206
    How Four-Second Clips Become Hours of Playable AI Soccer
  13. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
  14. 203
    The Thought a Model Doesn't Say — and the Lens That Reads It
  15. 197
    Twin Problems Suggest AI Reasoning Gains Are Mostly Better Fact Recall
  16. 179
    How DeepSeek Made One User Faster Without Slowing Down the Crowd
  17. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
  18. 158
    How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave
  19. 149
    When Cornering a Chatbot Makes It Lie: J.P. Morgan's Case for 'Playing Dead'
  20. 146
    How an Innocent README Can Freeze an AI Agent's Safety Check for an Hour
  21. 140
    When a Reasoning Model Says "Let Me Double-Check" After It's Already Decided
  22. 116
    Why Streaming Half a Reasoning Chain Beats Sending the Whole Thing
  23. 102
    How to Catch an AI Attack That No Single Conversation Reveals
  24. 094
    Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most
  25. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
  26. 077
    Reading a Model's Confidence Curve to Decide When Chain-of-Thought Is Worth It
  27. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
  28. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
  29. 041
    When the Iteration Teaches the Model to Skip the Iteration
  30. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
  31. 036
    Sparse Attention Was the Wrong Frame. Treat It as Geometry Instead.
  32. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
  33. 027
    When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure
  34. 018
    Language Models Compute the Rational Move, Then Override It

Related concepts

Entropy Gating Residual Stream Speculative Decoding Token-Level Analysis Variational Autoencoder

Related terms

commitment failure latency nucleus sampling speculative decoding token