forward pass

Definition

Plain language

One run of input through a neural network from start to finish to produce an output.

Why it matters: It's the basic unit of inference cost, and most performance and latency discussions ultimately reduce to how many forward passes you're doing.

For example, when you type a question to a chatbot, the model does one forward pass per output token to decide what word comes next.

Heard on the show

“Judging is one decision — one forward pass, one bit out.”

Mentioned in 18 episodes

1. 233
   Why a Model Can Grade an Answer But Not Write the Answer Key
2. 212
   The Fact Was in the Wrong Drawer: Why Fine-Tuned Models Can't Reason With What They Know
3. 185
   Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
4. 158
   How Floating-Point Rounding Lets a Model Tell Which Chip It's On — And Misbehave
5. 143
   When a Model Notices You Forged Its Own Words, And Why That Breaks Safety Tests
6. 128
   How a Model Can Earn Full Reward and Still Resist Training
7. 115
   Teaching a Phone Agent to Reason Silently, And Keeping It Honest
8. 085
   Why Long-Context Models Might Need Compute, Not Capacity, Before Eviction
9. 084
   Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
10. 074
    How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning
11. 060
    When Splitting One Model Across Three Agents Doubles Its Accuracy
12. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
13. 041
    When the Iteration Teaches the Model to Skip the Iteration
14. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
15. 027
    When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure
16. 018
    Language Models Compute the Rational Move, Then Override It
17. 013
    Why Search Keeps Rediscovering the Same Workflow, and What That Means
18. 003
    How to Pick the Best of Sixteen Coding Agent Rollouts

Related concepts

Related terms