Definition
Agentic RL applies reinforcement learning directly to multi-step, tool-using agent trajectories, training the model to take sequences of actions that lead to rewarded outcomes. It generalizes RLHF beyond single-turn responses, and brings classic RL headaches — credit assignment, exploration, reward hacking — into the LLM era.
Episodes covering this
Worth reading next
Papers we haven't done a deep dive on yet, but would recommend on this topic.
- WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning
- Is Reinforcement Learning (Not) the Solution to Robust Language Agent Tasks?
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- OpenHands: An Open Platform for AI Software Developers as Generalist Agents
- ELLM: Exploring with Large Language Models
- FunSearch: Making new discoveries in mathematical sciences using large language models
- Cognitive Architectures for Language Agents
- ExpeL: LLM Agents Are Experiential Learners
- TextGrad: Automatic 'Differentiation' via Text
- Reflexion: Language Agents with Verbal Reinforcement Learning
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
- WebArena: A Realistic Web Environment for Building Autonomous Agents
- World Models
- OpenDeepSearch: Democratizing Search with Open-source Reasoning Models
- Scaling LLM Test-Time Compute Optimally
- Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents
- OpenAI o1 System Card
- RLVR is Not RL: On the Importance of Grounding Reward Learning
- DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning
- Reinforcement Learning as Probabilistic Inference: A Survey
- AgentBench: Evaluating LLMs as Agents
- Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
- RL²: Fast Reinforcement Learning via Slow Reinforcement Learning
- AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents
- GiGPO: Group-in-Group Policy Optimization for LLM Agent Training
- RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
- GPTSwarm: Language Agents as Optimizable Graphs
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning