GRPO

Definition

Plain language

A reinforcement-learning recipe that compares several attempts at the same task to figure out which ones to reinforce.

Also called: G-R-P-O

Why it matters: Comparing rollouts within a group removes the need for a separate value network and is now the default RL recipe behind much of recent reasoning-model training.

For example, GRPO has a model produce eight attempts at the same math problem, then reinforces the ones that scored above the group average and discourages those below it.

Heard on the show

“Run GRPO with a judge-scored compliance reward.”

Mentioned in 27 episodes

1. 244
   The Open-Weight Defense That Feeds Attackers Confident, Falsified Answers
2. 199
   Finding a Model's Hidden Behaviors Without Knowing What You're Looking For
3. 193
   Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer
4. 186
   How a Frozen Model Went From 2% to 77% on Physics Puzzles — Without Retraining
5. 180
   The Bug Where Smart Assistants Read a Fact and Still Forget It
6. 166
   A Router That Beats the Frontier Models It Calls
7. 165
   A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants
8. 162
   The Empty-Lake Proof: Why More Rollouts Stop Helping Reasoning Models
9. 160
   Training an AI to Take Its Own Notes, So Its Future Self Works Better
10. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
11. 119
    Beating Reinforcement Learning Without Ever Touching the Model's Weights
12. 111
    How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations
13. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
14. 084
    Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
15. 082
    Training a Deep Research Agent on 8,000 Synthetic Tasks: The Rubric Tree Trick
16. 081
    When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence
17. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
18. 066
    Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
19. 064
    When Agent Memory Stops Being a Database and Starts Being a Skill
20. 059
    Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward
21. 052
    An Old Reinforcement Learning Tradeoff Sneaks Back Into LLM Agents
22. 051
    Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead
23. 048
    How a 30B Open Model Reached Olympiad Gold With the Right Recipe
24. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
25. 026
    What RL Actually Does to Language Models, at the Token Level
26. 011
    When RL Actually Teaches Agents Something New, And When It Doesn't
27. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training

Related concepts

Related terms