Glossary · Term

reinforcement learning

← all terms

Definition

Plain language

Training an AI by letting it try things and rewarding the attempts that work out.

As stated in the literature

A learning paradigm in which an agent optimizes a policy to maximize cumulative reward through interaction with an environment; the family encompassing PPO, GRPO, REINFORCE, and RLHF.

Also called: RL

Why it matters: It lets systems improve through trial and feedback rather than fixed examples, powering everything from game-playing agents to fine-tuning chatbots.

For example, an AI learning a game tries many moves and gradually favors the ones that earn the most points.

Heard on the show

“The signal is even there before any reinforcement learning — the poisoned training documents alone installed it.”
Episode 203 — The Thought a Model Doesn't Say — and the Lens That Reads It

Mentioned in 83 episodes

  1. 203
    The Thought a Model Doesn't Say — and the Lens That Reads It
  2. 199
    Finding a Model's Hidden Behaviors Without Knowing What You're Looking For
  3. 193
    Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer
  4. 189
    Why Phone Agents Ace the Test and Crash on Your Actual Phone
  5. 187
    An 8-Billion Agent That Beats Models 80 Times Its Size By Looking Things Up
  6. 186
    How a Frozen Model Went From 2% to 77% on Physics Puzzles — Without Retraining
  7. 183
    Why You Can't Fine-Tune Foresight Into an AI Agent
  8. 181
    How to Backpropagate Blame Through a Team of Chatbots — And When It Backfires
  9. 180
    The Bug Where Smart Assistants Read a Fact and Still Forget It
  10. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
  11. 173
    The Free Step-Level Grader Hiding in Every RL Training Run
  12. 171
    The Safety Decision a Model Makes Before It Thinks a Word
  13. 169
    Why Better Bug Reports Can Make AI Coding Agents Worse
  14. 167
    How Teaching an AI to Predict, Not Act, Made It a Better Actor
  15. 166
    A Router That Beats the Frontier Models It Calls
  16. 165
    A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants
  17. 163
    Why Training Only on Perfect Solutions Cripples a Model's Reasoning
  18. 162
    The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
  19. 160
    Training an AI to Take Its Own Notes, So Its Future Self Works Better
  20. 159
    Can a Coding Agent Run Its Own Robot Experiments Overnight, With No Human Resetting the Scene?
  21. 157
    When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed
  22. 155
    Why a Flawless Demo Makes a Worse Computer-Using Agent, And the Fix
  23. 154
    How a 7B Model Out-Investigates a 72B One by Choosing What to Look At
  24. 152
    Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good
  25. 148
    Why Letting an AI Watch Its Own Scoreboard Can Quietly Overwrite Its Safety
  26. 147
    Agents Fail at the Body, Not the Brain: A Self-Rewriting Scaffold That Lifts a 9B Model 44 Points
  27. 145
    Building Forgetting Into a Language Model With One Extra Line of Code
  28. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
  29. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
  30. 128
    How a Model Can Earn Full Reward and Still Resist Training
  31. 125
    AI Coding Agents Run a Marathon, and Fewer Than One in Three Finish
  32. 124
    A Cheap Model With the Blueprints Beats Expensive Models Working Blind
  33. 119
    Beating Reinforcement Learning Without Ever Touching the Model's Weights
  34. 118
    Why the Best-Aligned AI Models Are the Easiest to Trick Into Producing Harm
  35. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
  36. 114
    Agents That Rewrite Their Own Weights Instead of Just Taking Notes
  37. 111
    How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations
  38. 109
    An AI Got Caught Reading the Answer Key, And Why That Catch Matters
  39. 108
    The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks
  40. 107
    How a Market of Crippled AI Agents Outscored One Unrestricted Model
  41. 106
    Giving Agents a Notebook Instead of New Weights: How ExpGraph Lets Frozen Models Learn
  42. 104
    How Making a Research Agent Smarter Quietly Makes It Leak Your Secrets
  43. 099
    How an Open-Book Trick Teaches a Model to Catch Its Own Mistakes
  44. 096
    How Treating an AI Agent's Execution Like Git Recovers a Coordination Penalty
  45. 093
    A Calibrated Knob for Weak-to-Strong AI Oversight, Tested on Real Code
  46. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
  47. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
  48. 084
    Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
  49. 083
    Training the Translator: How a Small Communication Model Lets Agent Teams Outperform Themselves
  50. 082
    Training a Deep Research Agent on 8,000 Synthetic Tasks: The Rubric Tree Trick
  51. 081
    When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence
  52. 080
    How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
  53. 079
    An Old Idea From Cognitive Psychology Reshapes How We Reward Reasoning Models
  54. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
  55. 071
    When the Model Is Fine and the Plumbing Is Broken: Fixing Agents at the Interface
  56. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
  57. 069
    When Smarter Models Forecast Worse: The Hidden Failure Mode in LLM Predictions
  58. 068
    The OS Trick That Makes Tree Search Practical for Coding Agents
  59. 066
    Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
  60. 064
    When Agent Memory Stops Being a Database and Starts Being a Skill
  61. 060
    When Splitting One Model Across Three Agents Doubles Its Accuracy
  62. 059
    Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward
  63. 058
    Why Upgrading Your AI Auditor to a Smarter Model Can Make Your System Less Safe
  64. 055
    Why LLM Judges Flip Their Verdicts When You Change the Question Format
  65. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
  66. 052
    An Old Reinforcement Learning Tradeoff Sneaks Back Into LLM Agents
  67. 051
    Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead
  68. 048
    How a 30B Open Model Reached Olympiad Gold With the Right Recipe
  69. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
  70. 046
    When the AI Optimizer Edits the Grade Book: Why Harnessing Evolution Needs a Wall
  71. 028
    Teaching a Model to Hire Copies of Itself: Recursive Agent Optimization
  72. 026
    What RL Actually Does to Language Models, at the Token Level
  73. 025
    The Missing Gradient Term That Predicts Sycophancy in RLHF
  74. 022
    Training the Model Spec Directly: An Alignment Lever Aimed at the Say-Do Gap
  75. 021
    Ten Thousand Examples Beat the Full Industrial Pipeline for Search Agents
  76. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
  77. 018
    Language Models Compute the Rational Move, Then Override It
  78. 011
    When RL Actually Teaches Agents Something New, And When It Doesn't
  79. 010
    When Reward Climbs But Reasoning Goes Generic: Diagnosing Template Collapse in Agentic RL
  80. 009
    How Two Silent Library Bugs Quietly Invalidated a Wave of Reasoning Papers
  81. 008
    Why Long-Horizon AI Agents Get Stuck, and a Milestone-Based Fix That Helps
  82. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
  83. 003
    How to Pick the Best of Sixteen Coding Agent Rollouts

Related concepts

Agentic RL Credit Assignment DPO Entropy Regularization Exploration Hacking GRPO Loss Aggregation Math Reasoning Mixed-Policy Training Reward Variance Reward Variance RL Post-Training SNR-Aware Filtering Sparse Policy Selection Strategy Diversity Supervised Fine-Tuning Termination Poisoning Training Methods Trajectory Quality

Related terms

agent GRPO policy PPO REINFORCE RLHF