forward pass

Definition

Plain language

One run of input through a neural network from start to finish to produce an output.

Why it matters: It's the basic unit of inference cost, and most performance and latency discussions ultimately reduce to how many forward passes you're doing.

For example, when you type a question to a chatbot, the model does one forward pass per output token to decide what word comes next.

Heard on the show

“It touches one token position in a single forward pass and adds no new input.”

Mentioned in 16 episodes

1. 185
   Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
2. 158
   How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave
3. 143
   When a Model Notices You Forged Its Own Words, And Why That Breaks Safety Tests
4. 128
   How a Model Can Earn Full Reward and Still Resist Training
5. 115
   Teaching a Phone Agent to Reason Silently, And Keeping It Honest
6. 085
   Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction
7. 084
   Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
8. 074
   How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning
9. 060
   When Splitting One Model Across Three Agents Doubles Its Accuracy
10. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
11. 041
    When the Iteration Teaches the Model to Skip the Iteration
12. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
13. 027
    When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure
14. 018
    Language Models Compute the Rational Move, Then Override It
15. 013
    Why Search Keeps Rediscovering the Same Workflow, and What That Means
16. 003
    How to Pick the Best of Sixteen Coding Agent Rollouts

Related concepts

Related terms