Concept · 10 episode(s)

Computer-Use Agents

← all concepts

Definition

Computer-use agents drive a real computer interface — mouse, keyboard, screenshots — rather than calling structured APIs. They can in principle do anything a human can on a screen, which is exactly why they’re also the most worrying class of agent to deploy without good oversight.

Episodes covering this

  1. 189
    Why Phone Agents Ace the Test and Crash on Your Actual Phone
    Xiaomi-GUI-0 Technical Report
    Team, Qu, Luan · Xiaomi·24 min·Jul 02, 2026
  2. 185
    Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
    It Lied to a Doctor to Buy Poison Ingredients: Quantifying Real-World Misuse of Phone-use Agents
    Sun, Chen, Zhou et al. · Fudan University·27 min·Jun 30, 2026
  3. 157
    When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed
    Beyond the GUI Paradigm: Do Mobile Agents Need the Phone Screen?
    Gu, Jiang, Guo et al. · Mila–Québec AI Institute / Concordia University·24 min·Jun 19, 2026
  4. 156
    Why More Human Demonstrations Made a Computer-Use Agent Worse
    ProCUA-SFT Technical Report
    Jung, Lu, Cui et al. · NVIDIA / University of Washington·20 min·Jun 18, 2026
  5. 155
    Why a Flawless Demo Makes a Worse Computer-Using Agent, And the Fix
    Skill-Guided Continuation Distillation for GUI Agents
    Fan, Yu, Shen et al. · StepFun·22 min·Jun 18, 2026
  6. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
    MIRAGE: Mobile Agents with Implicit Reasoning and Generative World Models
    Yang, Hu, Hao et al. · Beihang University·24 min·Jun 04, 2026
  7. 080
    How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
    CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents
    Wang, Lu, Wang et al. · The University of Hong Kong·32 min·May 26, 2026
  8. 066
    Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
    ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agents
    Hu, Zhang, Xu et al. · Tongyi Lab·26 min·May 22, 2026
  9. 063
    Why Web Agents Are Slow: A Compiler-Style Fix for Computer-Use Latency
    Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling
    Winston, Wang, Mirhoseini et al. · Stanford University·26 min·May 21, 2026
  10. 017
    When the Agent Grades Its Own Homework: A Brutal New Benchmark for AI Workers
    Gym-Anything: Turn any Software into an Agent Environment
    Aggarwal, Neubig, Welleck · CMU·31 min·May 03, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.