Concept · 25 episode(s)

Rollout Sampling

← all concepts

Definition

Rollout sampling is the practice of generating many independent trajectories from a policy in order to estimate its expected behavior — reward, success rate, distribution over outcomes. It’s the workhorse evaluation technique for any non-deterministic policy.

Episodes covering this

  1. 179
    How DeepSeek Made One User Faster Without Slowing Down the Crowd
    DSpark: Confidence-Scheduled Speculative Decoding with
    XinCheng, XingkaiYu, ChenzeShao et al. · Peking University / DeepSeek-AI·23 min·Jun 27, 2026
  2. 172
    One Bad Token Can Sink a Model's Math, And You Can Delete It
    Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning
    Ko, Kang, Lee · Seoul National University·22 min·Jun 25, 2026
  3. 171
    The Safety Decision a Model Makes Before It Thinks a Word
    Do Thinking Tokens Help with Safety?
    Ri, Panigrahi, Arora · Princeton Language and Intelligence·25 min·Jun 25, 2026
  4. 165
    A Free-Lunch Tweak That Lets a Tiny Agent Beat Frontier Giants
    Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning
    Wang, Song, Zhang et al. · Peking University·22 min·Jun 23, 2026
  5. 160
    Training an AI to Take Its Own Notes, So Its Future Self Works Better
    Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
    Chen, Shi, Xie et al. · Alibaba Group·23 min·Jun 19, 2026
  6. 156
    Why More Human Demonstrations Made a Computer-Use Agent Worse
    ProCUA-SFT Technical Report
    Jung, Lu, Cui et al. · NVIDIA / University of Washington·20 min·Jun 18, 2026
  7. 141
    How Two Tokens Reopened a Reasoning Method the Field Had Given Up On
    Demystifying Hidden-State Recurrence: Switchable Latent Reasoning with On-Policy Reinforcement Learning
    Yang, Chen, Wu et al. · HKUST(GZ)·29 min·Jun 12, 2026
  8. 133
    How MiniMax Turned a Reward-Hacking Disaster Into Olympiad Gold
    MaxProof: Scaling Mathematical Proof with Generative-Verifier RL and Population-Level Test-Time Scaling
    Chen, Zhang, Zhang et al. · MiniMax / The Chinese University of Hong Kong·34 min·Jun 12, 2026
  9. 120
    How an AI Agent Rewrites Its Own Tools, Without an Answer Key
    Retrospective Harness Optimization: Improving LLM Agents via Self-Preference over Trajectory Rollouts
    Pan, Liu, Lin et al. · City University of Hong Kong·30 min·Jun 05, 2026
  10. 119
    Beating Reinforcement Learning Without Ever Touching the Model's Weights
    Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
    Hwang, Suri, Villecroze et al. · Layer6 AI·22 min·Jun 05, 2026
  11. 111
    How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations
    OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents
    Yang, Wu, Chen et al. · UIUC·24 min·Jun 03, 2026
  12. 099
    How an Open-Book Trick Teaches a Model to Catch Its Own Mistakes
    Self-Trained Verification for Training- and Test-Time Self-Improvement
    Wu, Raghunathan · Carnegie Mellon University·21 min·May 29, 2026
  13. 096
    How Treating an AI Agent's Execution Like Git Recovers a Coordination Penalty
    Shepherd: A Runtime Substrate Empowering Meta-Agents with a Formalized Execution Trace
    Yu, Chong, Nandi et al. · Northeastern University·22 min·May 28, 2026
  14. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
    The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
    MiniMax · MiniMax·28 min·May 27, 2026
  15. 088
    Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough
    SIA: Self Improving AI with Harness & Weight Updates
    Hebbar, Manawat, Verboomen et al. · Hexo Labs·25 min·May 27, 2026
  16. 081
    When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence
    Understanding and Mitigating Premature Confidence for Better LLM Reasoning
    Gai, Zeng, Baek et al. · Carnegie Mellon University·25 min·May 26, 2026
  17. 080
    How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
    CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents
    Wang, Lu, Wang et al. · The University of Hong Kong·32 min·May 26, 2026
  18. 068
    The OS Trick That Makes Tree Search Practical for Coding Agents
    DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback
    Dong, He, Hou et al. · Institute of Parallel and Distributed Systems·27 min·May 22, 2026
  19. 065
    One Loop to Optimize Them All: A Universal API for LLM-Driven Discovery
    optimize_anything: A Universal API for Optimizing any Text Parameter
    Agrawal, Lee, Tan et al. · UC Berkeley·27 min·May 22, 2026
  20. 063
    Why Web Agents Are Slow: A Compiler-Style Fix for Computer-Use Latency
    Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling
    Winston, Wang, Mirhoseini et al. · Stanford University·26 min·May 21, 2026
  21. 052
    An Old Reinforcement Learning Tradeoff Sneaks Back Into LLM Agents
    Look Before You Leap: Autonomous Exploration for LLM Agents
    Ye, Shi, Liu et al. · University of Science and Technology of China / Meituan·23 min·May 18, 2026
  22. 048
    How a 30B Open Model Reached Olympiad Gold With the Right Recipe
    Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling
    Li, Zhan, Zhang et al. · Shanghai AI Laboratory / The Chinese University of Hong Kong·31 min·May 16, 2026
  23. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
    Orchard: An Open-Source Agentic Modeling Framework
    Peng, Yao, Wu et al. · Microsoft Research·28 min·May 15, 2026
  24. 010
    When Reward Climbs But Reasoning Goes Generic: Diagnosing Template Collapse in Agentic RL
    RAGEN-2: Reasoning Collapse in Agentic RL
    Wang, Gui, Jin et al. · Northwestern University·22 min·May 02, 2026
  25. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
    Exploration Hacking: Can LLMs Learn to Resist RL Training?
    Jang, Falck, Braun et al. · MATS·23 min·May 02, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.