gradient

Definition

Plain language

The direction to nudge a model's weights to make it do better next time.

Also called: gradients

Why it matters: It's the central signal that drives essentially all neural-network training, and nearly every optimization technique is about computing or shaping gradients better.

For example, after a model misclassifies a cat photo as a dog, the gradient tells the optimizer which weights to nudge up or down to make 'cat' more likely next time.

Heard on the show

“And the reason for buckets instead of a precise number is the part I'd underline — the thing consuming this reward is a language model, not a gradient.”

Mentioned in 55 episodes

1. 186
   How a Frozen Model Went From 2% to 77% on Physics Puzzles — Without Retraining
2. 181
   How to Backpropagate Blame Through a Team of Chatbots — And When It Backfires
3. 177
   Why Raw Profiler Data Made an AI Worse at Writing GPU Code
4. 175
   One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
5. 167
   How Teaching an AI to Predict, Not Act, Made It a Better Actor
6. 166
   A Router That Beats the Frontier Models It Calls
7. 164
   The Summarizer That Quietly Deletes Your Agent's Safety Rules
8. 163
   Why Training Only on Perfect Solutions Cripples a Model's Reasoning
9. 162
   The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
10. 152
    Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good
11. 149
    When Cornering a Chatbot Makes It Lie: J.P. Morgan's Case for 'Playing Dead'
12. 148
    Why Letting an AI Watch Its Own Scoreboard Can Quietly Overwrite Its Safety
13. 145
    Building Forgetting Into a Language Model With One Extra Line of Code
14. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
15. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
16. 128
    How a Model Can Earn Full Reward and Still Resist Training
17. 127
    What Diffusion Language Models Were Missing: A Map, Not an Algorithm
18. 126
    How Coding Agents Can Mine Their Own Failures Into a Self-Targeting Curriculum
19. 119
    Beating Reinforcement Learning Without Ever Touching the Model's Weights
20. 118
    Why the Best-Aligned AI Models Are the Easiest to Trick Into Producing Harm
21. 117
    How an Open AI System Verified 672 Hard Math Proofs for Under $300
22. 114
    Agents That Rewrite Their Own Weights Instead of Just Taking Notes
23. 110
    How an Agent Got 44 Points Better by Mining Its Own Scratch Paper
24. 106
    Giving Agents a Notebook Instead of New Weights: How ExpGraph Lets Frozen Models Learn
25. 096
    How Treating an AI Agent's Execution Like Git Recovers a Coordination Penalty
26. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
27. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
28. 087
    When No Agent Reads the Whole Document: A Universal Cliff in Multi-Agent Review
29. 084
    Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
30. 082
    Training a Deep Research Agent on 8,000 Synthetic Tasks: The Rubric Tree Trick
31. 080
    How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
32. 074
    How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning
33. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
34. 067
    An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won
35. 065
    One Loop to Optimize Them All: A Universal API for LLM-Driven Discovery
36. 060
    When Splitting One Model Across Three Agents Doubles Its Accuracy
37. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
38. 053
    An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script
39. 051
    Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead
40. 048
    How a 30B Open Model Reached Olympiad Gold With the Right Recipe
41. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
42. 042
    An Agentic Scientific Computing System That Actually Remembers What It Learns
43. 041
    When the Iteration Teaches the Model to Skip the Iteration
44. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
45. 039
    When Smarter Agents Get Fooled by Three Extra Nodes in a Database
46. 038
    How LLMs Get Persuaded: One Attention Head, A Tetrahedron, And A Single Dial
47. 033
    Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval
48. 032
    A Sticky-Note for Every Layer: Letting Transformers Remember What They Were Just Thinking
49. 028
    Teaching a Model to Hire Copies of Itself: Recursive Agent Optimization
50. 025
    The Missing Gradient Term That Predicts Sycophancy in RLHF
51. 020
    The Compliance Gap: Why AI Says Yes and Does No
52. 010
    When Reward Climbs But Reasoning Goes Generic: Diagnosing Template Collapse in Agentic RL
53. 009
    How Two Silent Library Bugs Quietly Invalidated a Wave of Reasoning Papers
54. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
55. 001
    When AI Models Quietly Protect Each Other From Shutdown

Related concepts

Related terms