agent harness

Definition

Plain language

The wrapper of code, tools, and prompts around an AI agent that shapes how it actually behaves.

Also called: harness, harnesses

Why it matters: Headline 'model X is better' claims are often really harness comparisons, so understanding the harness is essential to evaluating any agent result.

For example, two teams using the same model but different harnesses — different tool definitions, parsers, and prompts — can get wildly different success rates on the same benchmark.

Heard on the show

“The design lets them separate the two — the same third-party harness ran three different models, and the same model ran under two harnesses.”

Mentioned in 17 episodes

1. 195
   Why 'Be Careful' Does Nothing for AI Coding Agents, and What Does
2. 174
   When the AI 'Schemes,' It's Usually Just Lazy or Confused
3. 166
   A Router That Beats the Frontier Models It Calls
4. 142
   Training a Tiny Model to Run the Plumbing Between an Agent and the World
5. 131
   Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix
6. 121
   When the Agent Says It's Done But Nothing Happened: Debugging the Harness, Not the Model
7. 120
   How an AI Agent Rewrites Its Own Tools, Without an Answer Key
8. 113
   What If a Prompt Injection Never Left? Attacks That Wait in Agent Memory
9. 105
   The Trojan Is Your Agent's Memory: Why Single-Step Defenses Miss Persistent Attacks
10. 097
    Same Tokens, Same Cost, Wildly Different Results: What Actually Scales in AI Agents
11. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
12. 078
    Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
13. 071
    When the Model Is Fine and the Plumbing Is Broken: Fixing Agents at the Interface
14. 061
    When Helpful Agents Go Sideways: A 404 Error, Campus Security, and Why Alignment Misses This
15. 047
    When Agent Benchmarks Lie: The Harness Problem in Open-Source AI
16. 014
    Why a Constrained Pipeline Beat a Full Coding Agent at Finding Bugs 30-to-1
17. 001
    When AI Models Quietly Protect Each Other From Shutdown

Related concepts

Related terms