Tool use: what happens when agents call calculators, shells, and APIs

The episodes (51)

Newest first. Each line is what that paper contributed to the question.

• The Tool Description Was the Attack: How Agents Leak Their Own Context

  The attack exploits how agents select and populate arguments for external tools/functions in a tool-calling loop. (0 out of 200 malicious tools flagged)

• Stealing an AI Agent's Expertise Without Copying a Word of It

  Hidden skills include tool preconditions and output schemas that agents invoke, which the attack tries (and fails) to recover exactly. (~87% of the capability gap closed in ~30 queries)

• The Same Model Refused a Backdoor, Then Its Own Sub-Agent Ran It

  Attacker content enters via tool outputs like files and shell command results that the agent reads and acts on. (13 out of 13 on all six harnesses)

• Poisoned Bug Reports Fooled Coding Agents Two Times Out of Three

  The agents' ability to run shell commands and install packages is exactly what the injected instructions exploit. (66.5% of attacks succeeded)

• The AI Agent That Found the Truth and Typed the Lie Anyway

  The agent's search tool calls and when it decides to stop calling them is central to the failure mechanism. (96% to 26% from one fake page)

• The Bias Isn't in Your Prompt — It's Inside the Model

  Coin-flip and timestamp randomness tools are used to test whether models cheat even 'fair' external randomness. (6.5 points lower on its own maker's bubble odds)

• Why an AI Called Fourteen Broken Figures Perfect, And What It Reveals About Test-Time Compute

  Grounded instruments like linters, code execution, and geometry tools drive the interaction loop's fixes. (14 of 15 rated perfect, only 3 actually clean)

• The Blank Space in Your AI Approval Box That Isn't Empty

  The attack exploits how MCP tool descriptions are fed into an AI assistant's context as trusted instructions. (1 of 8 attacks stayed fully invisible)

• How Do You Know an AI Agent Actually Refused? Check the World, Not the Words

  A monitored tool gateway records real tool outputs versus what the agent is shown, central to detecting deception. (94%)

• Why 'Be Careful' Does Nothing for AI Coding Agents, and What Does

  Coding agents with shell access execute destructive real-world actions based on vague natural-language instructions. (65.5% vs 64% action rate — sandbox vs production cues)

• How a Robot Builds a Debugging Notebook It Can Read, Edit, and Hand to Another Robot

  Robot programs stitch together pre-built subroutines (perception, motion planning, grasping) as callable tools via code-as-policy. (31% vs 4%)

• A 32B Open Model Matched Frontier Systems By Learning to Take Notes

  Memory operations (read, write, search, upsert) are elevated to first-class agent actions interleaved with task actions. (2x-4x gains, same weights)

• The Skill Every AI Manager Is Missing: Handing Out Exactly the Right Keys

  A key management axis is whether the orchestrator assigns the right subset of tools to each helper agent. (under 50% permission precision)

• An 8-Billion Agent That Beats Models 80 Times Its Size By Looking Things Up

  The paper's central finding is that training an agent to actively use medical tools beats larger models that merely have access to tools. (8B beats 671B by 15+ points)

• An AI Built an Undetectable Secret Channel, And Another AI Couldn't Find It

  Tool access—web search, filesystem, a helper model for token probabilities—is what makes the construction barrier collapse.

• How to Backpropagate Blame Through a Team of Chatbots — And When It Backfires

  The τ-bench retail task requires agents to make correct tool calls as part of evaluation. (71 to 7)

• One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent

  Tool-calling accuracy is the target behavior the paper tries to localize, steer, and transfer via crosscoder features. (+65 percentage points from one feature)

• When a One-Liner Beats Your Agent's Clever Verification Logic

  The paper models a menu of tools (syntax checker, test suite, LLM critic, oracle) with different costs and reliabilities. (670-second verification)

• Why Better Bug Reports Can Make AI Coding Agents Worse

  SHERLOC uses a fixed four-tool menu (view file, search, repo tree, connected tree) as the agent's only interface to the codebase. (74% to 10%)

• When Turning Experience Into Code Makes Your AI Agent Dumber

  A central comparison is between storing experience as callable code tools versus natural-language text, with tools shown to fail under distribution shift. (53% vs 63%)

• How Teaching an AI to Predict, Not Act, Made It a Better Actor

  World-model warm-up with zero tool-calling data transferred to multi-turn tool-use benchmarks, improving performance by ~9 points on average. (9 points better on an unseen benchmark)

• A Robot That Plays Before You Give It a Job, And Why That Beats Retrying

  The Code-as-Policy agent calls structured perception, grasping, and motion tools as building blocks of its generated programs.

• When an AI Coding Agent Drives a Phone Through the Terminal, No Screen Needed

  Custom database-query and file-access tools are added to the terminal agent, and the paper studies how tool benefit varies with underlying model strength.

• Don't Kill the Loser: A Different Way to Handle Two AI Agents Colliding

  The ToolSmith component dynamically generates footprint-declared, undoable tools for agents operating on bare systems.

• Agents Fail at the Body, Not the Brain: A Self-Rewriting Scaffold That Lifts a 9B Model 44 Points

  The Wikipedia repair story exemplifies runtime tool generation: the system writes and deploys a new retrieval tool autonomously mid-run.

• When an AI Agent Just Copies Its Tool — And Bigger Models Copy More

  The entire paper examines how LLM agents invoke and over-trust a GNN specialist tool for node classification.

• Training a Tiny Model to Run the Plumbing Between an Agent and the World

  The harness operates on generic tool-use trajectories — intercepting and filtering shell commands and environment observations between the LLM and the real environment.

• Five Identical Worlds, One Swapped Model: What Happens When AI Agents Run for Fifteen Days

  Agent capabilities are gated by location and preconditions at runtime, making certain tools literally uncallable rather than merely discouraged.

• How an AI Agent Rewrites Its Own Tools, Without an Answer Key

  The optimized harness includes new executable scripts/tools the agent writes for itself, going beyond memory to actionable machinery.

• How an Agent Got 44 Points Better by Mining Its Own Scratch Paper

  The paper introduces 'pseudo-tools' — LLM-backed callable tools with natural-language descriptions extracted from agent traces.

• The Reasoning Cliff: Why Thinking Longer Makes Models Worse at Exact Step-by-Step Tasks

  The main prescription is to delegate past the 'deterministic horizon' to exact algorithmic tools rather than continuing to reason neurally.

• The Trojan Is Your Agent's Memory: Why Single-Step Defenses Miss Persistent Attacks

  The agent harness gives models file read/write and shell access, which are the channels through which the attack plants and triggers backdoors.

• How Making a Research Agent Smarter Quietly Makes It Leak Your Secrets

  The agents' primary mechanism is issuing queries to a search engine as an external tool, and the privacy risk arises directly from that tool-use pattern.

• How a Prompt Wrapper Lets a Frontier Model Play Poker Like an Expert

  Deterministic code computes hard game-state facts (hand class, pot odds, stack depth) that are fed to the model, preventing hallucination.

• Same Tokens, Same Cost, Wildly Different Results: What Actually Scales in AI Agents

  The metric scores tool-call results on validity and non-redundancy, making tool-use quality a first-class variable in the scaling framework.

• When AI-Written Papers Read Well But the Evidence Underneath Is Broken

  DeepScientist was instructed to call the Semantic Scholar API for citation verification but never did, illustrating tool-use failures in agentic pipelines.

• An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won

  Agent B and D variants call AlphaProof as an external specialized tool for subgoal proving.

• Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer

  The central experiment is adding structured tool calls to a GUI agent's action space and studying the performance impact.

• Why Web Agents Are Slow: A Compiler-Style Fix for Computer-Use Latency

  Cached browser tools with typed preconditions and postconditions are the building blocks of the compiled agent programs.

• Treating Hallucinations as Exploits: A Gate-Based Architecture for Agent Safety

  The architecture centers on constraining which tool-invocation actions can be authorized based on certified evidence rather than model-generated claims.

• Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward

  The paper focuses entirely on training models to make verified sequences of tool/API calls.

• How Uber Caught 206 Leaked Credentials With an LLM-Powered Security Stack

  MCP-connected tool calls are the primary surface area for attacks ADR is designed to detect.

• Two Frozen Models Learn to Whisper: Coupling Through Hidden States

  The auxiliary model has access to tools (calculator, Z3 solver, Python) and the architecture enables the primary to benefit from their results.

• When Smarter Agents Get Fooled by Three Extra Nodes in a Database

  The vulnerability exploits the agent's trusted tool-call channel (MCP) to deliver poisoned structured data as observational ground truth.

• Why Frontier Agents Ask for Clarification at Exactly the Wrong Moment

  Agents are evaluated on tool-use benchmarks and use an ask-the-user tool whose invocation timing is the central variable.

• Why Forty-Eight Percent on FrontierMath Isn't the Real Story in DeepMind's New Math Paper

  The system uses coding agents to run Python (e.g., PySAT) and literature search tools as part of its problem-solving.

• An AI Agent That Found 28 Zero-Days in Windows — And What Made It Work

  slyp uses three specialized tool servers (binary exploration, COM inspection, dynamic debugging) that the LLM calls to do grunt work.

• Ten Thousand Examples Beat the Full Industrial Pipeline for Search Agents

  Expanding tool diversity in training demonstrations is one of the three core data modifications studied in the paper.

• The Compliance Gap: Why AI Says Yes and Does No

  The gap is operationalized and measured through tool-call logs, making tool-use behavior the ground-truth channel for actual compliance versus verbal compliance.

• Why Your Coding Agent Stalls While the GPU Runs Hot

  Tool calls create CPU load spikes and GPU idle periods that MARS must jointly schedule around, making tool-use patterns central to the problem.

• When RL Actually Teaches Agents Something New, And When It Doesn't

  Agents use a search tool across multiple interaction rounds, making tool-use the key differentiator from pure math reasoning benchmarks.