Concept · 27 episode(s)

GRPO

← all concepts

Definition

GRPO (Group Relative Policy Optimization) is a policy-gradient method that estimates advantage by comparing a sampled response to the average of a group of other samples from the same prompt, removing the need for a separate value model. It became a popular RL-from-rewards variant in reasoning-model training pipelines.

Episodes covering this

  1. 199
    Finding a Model's Hidden Behaviors Without Knowing What You're Looking For
    Mechanistically Eliciting Latent Behaviors in Language Models
    Mack, Panickssery, Turner · Principles of Intelligence·15 min·Jul 04, 2026
  2. 193
    Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer
    Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
    Zhang, Hu, Glentis et al. · University of Minnesota·22 min·Jul 02, 2026
  3. 186
    How a Frozen Model Went From 2% to 77% on Physics Puzzles — Without Retraining
    Hierarchical Experimentalist Agents
    Chandra, Vaidyanathan, Dhanuka et al. · University of Massachusetts Amherst·22 min·Jun 30, 2026
  4. 180
    The Bug Where Smart Assistants Read a Fact and Still Forget It
    Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents
    Patel · Vrin·24 min·Jun 29, 2026
  5. 166
    A Router That Beats the Frontier Models It Calls
    Sakana Fugu Technical Report
    Tang, Cetin, Xu et al. · Sakana AI·26 min·Jun 23, 2026
  6. 165
    A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants
    Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning
    Wang, Song, Zhang et al. · Peking University·22 min·Jun 23, 2026
  7. 160
    Training an AI to Take Its Own Notes, So Its Future Self Works Better
    Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
    Chen, Shi, Xie et al. · Alibaba Group·23 min·Jun 19, 2026
  8. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
    Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
    Yang, Chen, Wu et al. · HKUST(GZ)·29 min·Jun 12, 2026
  9. 119
    Beating Reinforcement Learning Without Ever Touching the Model's Weights
    Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
    Hwang, Suri, Villecroze et al. · Layer6 AI·22 min·Jun 05, 2026
  10. 111
    How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations
    OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents
    Yang, Wu, Chen et al. · UIUC·24 min·Jun 03, 2026
  11. 109
    An AI Got Caught Reading the Answer Key, And Why That Catch Matters
    EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
    Chen, Shi, Li et al. · Shenzhen Institutes of Advanced Technology·28 min·Jun 03, 2026
  12. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
    SIA: Self Improving AI with Harness & Weight Updates
    Hebbar, Manawat, Verboomen et al. · Hexo Labs·25 min·May 27, 2026
  13. 084
    Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
    ECHO: Terminal Agents Learn World Models for Free
    Shrivastava, Kauffmann, Awadallah et al. · Microsoft Research·26 min·May 26, 2026
  14. 082
    Training a Deep Research Agent on 8,000 Synthetic Tasks: The Rubric Tree Trick
    QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks
    Xie, Lin, Wang et al. · The Ohio State University·31 min·May 26, 2026
  15. 081
    When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence
    Understanding and Mitigating Premature Confidence for Better LLM Reasoning
    Gai, Zeng, Baek et al. · Carnegie Mellon University·25 min·May 26, 2026
  16. 079
    An Old Idea From Cognitive Psychology Reshapes How We Reward Reasoning Models
    Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals
    Chen, Xu, Zhao et al. · Tongji University / Shanghai AI Laboratory / Nanyang Technological University·29 min·May 25, 2026
  17. 073
    When Three LLMs Talk to Each Other, Their Ideas Quietly Stop Moving
    Multi-LLM Systems Exhibit Robust Semantic Collapse
    Kong, Lai, Piao et al. · University of Toronto·28 min·May 23, 2026
  18. 066
    Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
    ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agents
    Hu, Zhang, Xu et al. · Tongyi Lab·26 min·May 22, 2026
  19. 064
    When Agent Memory Stops Being a Database and Starts Being a Skill
    Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents
    Ye, Liu, Wang et al. · University of Illinois Urbana-Champaign·30 min·May 22, 2026
  20. 059
    Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward
    Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs
    Lu, Wang, Lu et al. · Northeastern University·22 min·May 20, 2026
  21. 052
    An Old Reinforcement Learning Tradeoff Sneaks Back Into LLM Agents
    Look Before You Leap: Autonomous Exploration for LLM Agents
    Ye, Shi, Liu et al. · University of Science and Technology of China / Meituan·23 min·May 18, 2026
  22. 051
    Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead
    Argus: Evidence Assembly for Scalable Deep Research Agents
    Zhang, Su, Chen et al. · MiroMind AI·22 min·May 18, 2026
  23. 048
    How a 30B Open Model Reached Olympiad Gold With the Right Recipe
    Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling
    Li, Zhan, Zhang et al. · Shanghai AI Laboratory / The Chinese University of Hong Kong·31 min·May 16, 2026
  24. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
    Orchard: An Open-Source Agentic Modeling Framework
    Peng, Yao, Wu et al. · Microsoft Research·28 min·May 15, 2026
  25. 026
    What RL Actually Does to Language Models, at the Token Level
    Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning
    Akgül, Kannan, Neiswanger et al. · University of Southern California·24 min·May 08, 2026
  26. 011
    When RL Actually Teaches Agents Something New, And When It Doesn't
    Does RL Expand the Capability Boundary of LLM Agents? A PASS@(k,T) Analysis
    Zhai, Yan, Shao et al. · Fudan University·23 min·May 02, 2026
  27. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
    Exploration Hacking: Can LLMs Learn to Resist RL Training?
    Jang, Falck, Braun et al. · MATS·23 min·May 02, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.