Agent scaffolding: why the harness matters as much as the model

The episodes (41)

Newest first. Each line is what that paper contributed to the question.

• The Same Model Refused a Backdoor, Then Its Own Sub-Agent Ran It

  The harness that assembles model context (system/user/tool roles) is the actual site of the vulnerability, not the model. (13 out of 13 on all six harnesses)

• Why 'Be Careful' Does Nothing for AI Coding Agents, and What Does

  Comparing the same model across first-party vs third-party harnesses shows the harness, not the model, determines whether hesitation becomes a clarifying question. (65.5% vs 64% action rate — sandbox vs production cues)

• A 32B Open Model Matched Frontier Systems By Learning to Take Notes

  Loop one iteratively rewrites the agent's memory tool scaffold—prompts, file formats, and operations—using a frontier model reviewer. (2x-4x gains, same weights)

• A Coding Agent Found a Hole in a Peer-Reviewed STOC Proof for Five Dollars

  An orchestrator agent manages sub-tasks, parallel candidates, backtracking, and specialist helpers throughout the formalization pipeline. (1 line)

• How to Backpropagate Blame Through a Team of Chatbots — And When It Backfires

  The method operates on fixed-backbone agents whose only tunable component is their prompt instructions. (71 to 7)

• A Router That Beats the Frontier Models It Calls

  The episode extensively discusses how software harnesses (Claude Code, Codex) wrapped around models dramatically amplify their effective capability. (~5-6% relative gain on agentic coding)

• The Summarizer That Quietly Deletes Your Agent's Safety Rules

  Real production frameworks — LangGraph, LangMem, AutoGen — are shown to reproduce the failure, making context compaction a first-class governance surface in agent scaffolding. (0% to 59% violation rate)

• When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed

  The terminal agent relies on a carefully hand-crafted system prompt with a four-phase reasoning cycle, priority hierarchy, and custom tools as its scaffolding.

• Agents Fail at the Body, Not the Brain: A Self-Rewriting Scaffold That Lifts a 9B Model 44 Points

  The paper's central contribution is a composable, evolvable agent harness that treats scaffolding as a first-class optimizable object.

• How an Innocent README Can Freeze an AI Agent's Safety Check for an Hour

  The paper targets the guardrail layer sitting in agent scaffolding architectures, showing how safety wrappers become attack surfaces.

• Training a Tiny Model to Run the Plumbing Between an Agent and the World

  The paper's central subject is the harness (scaffold) — the software wrapper between an LLM and the environment — and whether it can be learned rather than hand-coded.

• When Optimizing One GPU Kernel Quietly Breaks the Whole System

  The central thesis is that the bottleneck for autonomous system optimization is structure — the harness around the model — not model intelligence.

• The Agent Failed — But Did the Instructions Deserve to Be Followed?

  Skill documents are runtime instruction artifacts injected into agent context, a form of scaffolding for guiding agent behavior.

• Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix

  Arbor's hypothesis tree, coordinator/executor split, and merge gate constitute an elaborate scaffolding for autonomous research.

• How a Crowd of Anonymous AI Agents Broke a 40-Year Math Record

  The paper contrasts bespoke per-problem scaffolds (AlphaEvolve paradigm) with their shared persistent platform-as-harness design.

• AI Coding Agents Run a Marathon, and Fewer Than One in Three Finish

  The scaffold (harness) is shown to matter as much as the model, causing up to 12x variation in token usage across the same model.

• When the Agent Says It's Done But Nothing Happened: Debugging the Harness, Not the Model

  The paper's central argument is that most agent failures live in the harness/scaffolding around the model, not in the model itself.

• How an AI Agent Rewrites Its Own Tools, Without an Answer Key

  The paper's central object of study is the 'harness' — the system prompts, tool definitions, and scripts wrapping a frozen model — which is exactly agent scaffolding.

• How an Agent Got 44 Points Better by Mining Its Own Scratch Paper

  The induced reasoning-primitive library functions as automatically discovered scaffolding around a frozen base agent.

• How a Market of Crippled AI Agents Outscored One Unrestricted Model

  The paper explicitly contrasts its market-driven approach against traditional orchestrator-based scaffolding, arguing central orchestrators are bottlenecks.

• How to Catch an AI Attack That No Single Conversation Reveals

  The attack relies on an external orchestrator scaffold that coordinates subagents and holds the full harmful objective outside any single model context.

• How a Prompt Wrapper Lets a Frontier Model Play Poker Like an Expert

  PokerSkill wraps LLMs in a structured scaffold — context engine, skill retrieval, budget system — that is the paper's core architectural contribution.

• Same Tokens, Same Cost, Wildly Different Results: What Actually Scales in AI Agents

  The harness (scaffolding) is the unit of analysis; different harness families are compared to determine how efficiently they convert budget into useful feedback.

• How Treating an AI Agent's Execution Like Git Recovers a Coordination Penalty

  Shepherd is explicitly a runtime substrate — scaffolding infrastructure — that exposes first-class primitives for observation, replay, and branching of agent execution.

• A Calibrated Knob for Weak-to-Strong AI Oversight, Tested on Real Code

  The calibrated conservatism controller wraps around an existing agent at deployment time without retraining.

• When Better Fine-Tuning Can't Help: A Geometric Impossibility in LLM Causal Reasoning

  A-CBO wraps a frozen LLM in an external Bayesian decision loop, demonstrating that scaffolding architecture — not model weights — drives the accuracy gain.

• Two Levers for Self-Improving AI: When Rewriting Code Isn't Enough

  A Meta-Agent writes and iteratively rewrites the task agent's scaffold code as one of two self-improvement levers.

• Same Model, Organized Differently: How an Agent Architecture Beat Frontier Systems at Research Math

  Wrapping Claude Opus 4.6 in RMA's scaffolding lifts it from zero to eight out of ten solved problems.

• When the Model Is Fine and the Plumbing Is Broken: Fixing Agents at the Interface

  The paper's central contribution is a structured harness layer — scaffolding — that sits between the model and environment to catch and correct failures.

• Why Web Agents Are Slow: A Compiler-Style Fix for Computer-Use Latency

  The JIT compilation framework is a novel scaffolding architecture replacing the standard screenshot-and-decide interpreter loop.

• Treating Hallucinations as Exploits: A Gate-Based Architecture for Agent Safety

  The evidence-carrying architecture defines a specific scaffolding pattern: typed certificates from narrow verifiers, action schemas, and a deterministic gate outside the model.

• When Helpful Agents Go Sideways: A 404 Error, Campus Security, and Why Alignment Misses This

  Four different agent frameworks/harnesses are tested, and the paper shows meltdowns are universal across them.

• An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script

  The AIRA-dojo harness provides the tree-based search structure (Draft/Debug/Improve/Analyze operators) within which agents iterate.

• Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead

  The Navigator/Searcher separation and shared evidence DAG schema are architectural scaffolding outside any single model's context.

• When Agent Benchmarks Lie: The Harness Problem in Open-Source AI

  The episode's central argument is that the harness/scaffolding layer fundamentally determines what agent capability is real versus memorized.

• When the AI Optimizer Edits the Grade Book: Why Harnessing Evolution Needs a Wall

  The harness — a locked workspace with a CLI gateway and forbidden-action list — is the scaffolding that keeps the meta-agent's edits safe.

• Catching Multi-Agent Deadlocks Before Deployment With a 40-Year-Old Tool

  The topology monitor and verified protocol form the structural scaffolding constraining each agent's coordination behavior at runtime.

• Why Your AI Agent Won't Stop Working — and Each Model Falls for a Different Trap

  The paper argues for external scaffolding — step ceilings, provenance tracking, external validators — as the durable defense against termination poisoning.

• When AI Agents Build the Serving Stack: A Bet on Bespoke Infrastructure

  The nested outer-planner / inner-trio architecture with role separation and structured artifact handoffs is the paper's primary engineering contribution.

• An AI Agent That Found 28 Zero-Days in Windows — And What Made It Work

  The paper's central empirical finding is that the tool scaffold — binary explorer, COM inspector, live debugger — drives exploit success more than the underlying model.

• Why AI Coding Agents Keep Trying to Debug Without a Debugger

  DAIRA adds a three-phase workflow (reproduce, trace-diagnose, patch-verify) as orchestration around SWE-agent.