GUI

Definition

Plain language

The clickable, windowed interface most computer programs show their users.

Why it matters: Most software in the world only has a GUI, so agents that can drive them open up automation for tasks that don't expose APIs.

For example, a computer-use agent has to click the right button in a desktop app's GUI, which means it needs to actually see and interpret the screen.

Heard on the show

“And here's why this matters even if you don't follow GUI agents at all.”

Mentioned in 9 episodes

1. 189
   Why Phone Agents Ace the Test and Crash on Your Actual Phone
2. 167
   How Teaching an AI to Predict, Not Act, Made It a Better Actor
3. 157
   When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed
4. 155
   Why a Flawless Demo Makes a Worse Computer-Using Agent, And the Fix
5. 084
   Terminal Agents Get Free Supervision From The Tokens We've Been Throwing Away
6. 080
   How a Two-Agent Trick Unlocked Large-Scale Training for Computer-Use Agents
7. 066
   Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
8. 047
   When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
9. 017
   When the Agent Grades Its Own Homework: A Brutal New Benchmark for AI Workers

Related terms