All episodes
Episode 295 · Nov 16, 2026 · 14 min

A Strong Model Delegates Its Reading, And Starts Believing Forgeries

Wang, Mannby

AI Agent Security
PaperDive — Episode 295: A Strong Model Delegates Its Reading, And Starts Believing Forgeries — cover art
paperdive.ai

Point at a folder with five forged documents and it catches the fake 179 times out of 180. Let it hand the reading to the cheaper helper model it ships with — same model on top, same files — and it adopts the forger's answer 75 times out of 180. A new paper argues the model you trust only protects you on the evidence it reads itself.

Key takeaways

  • Why a strong that catches a forgery 180 out of 180 times starts failing once a cheaper reads the files for it
  • The two controls that rule out 'summaries just lose information': a clean folder gave zero failures, and an equal-strength helper gave near-zero
  • The uncomfortable detail inside the handoff — even when the helper's report did flag the forgery, the sided with the fake 21 times out of 96
  • Why explicit was resisted everywhere, while forged text that 'makes no demand' got through
  • The honest limits: forgeries were built to be catchable, failure means picking a wrong fictional name, and the 'weakest ' slogan doesn't hold as a clean rule
  • Two cheap defenses that brought failures from 15 to 3 and 1 in a 40-trial test — including one that just tells a to be skeptical

Our reservations

Where the slogan breaks down. Eric pushes back on the title's claim: the system failed 75 times where the cheap model alone failed 136, so it inherits an rather than the weak reader's specific errors — plus noise and delivery-route effects. listen from 09:32

Ep. 295
A Strong Model Delegates Its Reading, And Starts Believing Forgeries
0:00
14 min
Paper
Only as Safe as the Weakest Agent: A Break That Appears Only When the System Is Multi-Agent
Venue
arXiv:2611.17520
Year
2026
Read the paper
arxiv.org/abs/2611.17520
Also available on
Apple Podcasts Spotify

Chapters

  1. 00:00An attack that doesn't work — until it does
  2. 01:20Where the evidence stops being the model's
  3. 02:15Forgeries designed to be catchable
  4. 03:13What 'failure' actually means here
  5. 04:01The numbers jump everywhere
  6. 04:58Is it the handoff, or the reader?
  7. 06:19The warning that never arrives — or gets ignored
  8. 07:43Why delegation removes the occasion to verify
  9. 08:23The forgery that makes no demand
  10. 09:32Our reservations: where the slogan breaks down
  11. 10:39Give it the source, errors go to zero
  12. 11:58The cheap helper is a trust boundary

References in this episode

Also available as a plain-text transcript page.

0:00Paige: Picture a folder of 35 files. Most are genuine records, and five are forgeries, all pushing the same false fact. Researchers pointed at folders like that, with 's 5 in charge, and asked a simple question the real records answer. In all 180 trials, the model called out the forgery's missing paperwork. It fell for the fake once.

0:23Eric: Which sounds like a paper about an attack that doesn't work.

0:27Paige: It does, until they change one thing. Instead of reading the files itself, the same model hands the reading to the cheaper helper model, that as a built-in . The facts are the same, and the forged files are the same. Now it gives the forger's answer 75 times out of 180.

0:45Eric: From one to seventy-five, and nobody touched the model. Okay, that's a paper about something.

0:52Paige: This is AI Papers: A Deep Dive. Today's paper is "Only as Safe as the Weakest Agent," by Keegan Wang and Anantika Mannby. Their affiliations are Carnegie Mellon, the University of Southern California, and . It was posted on November 12th, 2026, and it's a accepted to a on that manage other agents. Our question is this: when you build a system around a very capable model, does the system inherit that model's judgment?

1:21Eric: A of background first, because isn't just a model. A coding is a model inside a loop. It gets a task, and it calls a tool, like reading a file or searching a directory. The result comes back as text, and the model decides what to do next. Everything it knows about your disk arrives through those calls.

1:41Paige: And both and 's let the main model, called the , a helper called a to do a of work and report back. The paper says both products recommend this for searching across many files. It also says that cheaper models, in many cases, cost about a tenth as much per .

2:02Eric: So this is the economical, recommended way to run them. The side effect is structural. The model that reads the evidence and the model that makes the decision, they're no longer the same model.

2:14Paige: That split is the experiment. Each test item is a short fictional passage. The paper's example says Kessa Lorn designed the Velra compact, and Kessa Cole surveyed the ferry road next to it. Then the question is, who designed the compact? Each item has two candidate names. There are 180 items, all invented, so no model can lean on knowledge.

2:36Eric: And what do the forgeries look like?

2:39Paige: They look like genuine paperwork, which is exactly the point, and they come in a few styles. One is a notice claiming the record was amended. Another is an annotation signed by a "senior archivist," saying the record was misread. The genuine record always stays in the folder, so the truth is recoverable. And every forgery has the same two flaws, fixed in advance. It's dated before the record it claims to correct. It's also missing the , which is a catalog entry that every real record carries.

3:10Eric: So a careful reader catches it with boring checks. I want to flag the scoring now, before the numbers get exciting. Failure here is what they call a . The answers correctly on a clean folder, then gives the forger's answer once the forgery is present. That means it picked the wrong fictional name. Nobody ran malicious code, and nothing was stolen. The title runs hotter than the measurement.

3:36Paige: That's fair, and the paper is clear about its measure. Within that measure, the is stark. They tested , , and a small , a test setup the authors wrote themselves. They looked at forgery styles those could catch on their own. Reading the files directly produced two in 1,080 trials. Other styles beat them even without any at all.

4:01Eric: Then comes . How did they force it?

4:04Paige: They forced it two different ways, depending on the product. On , they forced it mechanically. A command-line setting removes the 's file tools, so the 's report is the only evidence it ever gets. has no equivalent switch, so there they instructed it to and checked each trial.

4:25Eric: That's a softer guarantee, and the paper gives no compliance rate for it. What happened?

4:31Paige: Even with that softer guarantee, the failures jumped in every setup. With the archivist annotation, went from 1 failure in 180 to 20 in 180. had no failures reading directly, but 44 in 180 after . The five-forgery version on Claude Code produced our 75 failures out of 180. In their own , they used a forgery style built on repetition. Pooling the results over three , they got 553 failures out of 900.

4:59Eric: My first suspicion is that this is just summaries losing information. Maybe handing anything to anyone creates errors, and the forgery is incidental.

5:08Paige: They tested that, and this is where the paper earns its claim. The first control over a clean folder with no forgery. It produced zero out of 180 in each product. The second control makes the helper the same strong model as the . On , that gave zero harmful revisions in 180 trials. had one in 180, and their had 11 in 540.

5:32Eric: Oh, that's clean. The handoff alone is harmless, and a handoff to an equal is nearly harmless. It only breaks when the reader is weaker.

5:41Paige: And across eight pairings in their , the damage grows steadily as the helper gets weaker.

5:48Eric: There's one more , though. When the , it writes the helper's instructions itself. It tends to ask for a summary of what the files say, not a choice between two names. So the question changed, not only the reader.

6:03Paige: They designed a control for that, where the helper gets the direct prompt word for word. But neither product lets you rewrite the 's prompt, so that control could only run in their own . For and specifically, that one is argued, not tested. The part I keep coming back to is what they found inside the handoff. Claude Code exposes the sub-agent's report, so they could read what the received. When it went through the files itself, the orchestrator called out the missing documentation in all 180 trials. The helper's report carried that warning only 96 times.

6:40Eric: So nearly half the time, the warning never arrives. It's the senior lawyer who'd spot a forged contract in seconds, except they never open it, because an intern summarized the box and the memo left out the bad date.

6:54Paige: And the data pushes your analogy further than it wants to go. Even when the memo did flag the problem, the still sided with the forgery 21 times out of 96.

7:05Eric: Wait, it had the warning right there? And that's against once in 180 when it went through the record itself?

7:12Paige: It is, with the warning sitting right there in the report. The model failed once in 180 when it handled the evidence itself, and 21 times in 96 when the warning came secondhand. The strong model isn't only a victim of a bad summary.

7:27Eric: A warning in someone else's memo is a weaker object than a warning you noticed yourself. That bothers me more than the main result. Although nothing randomized what the helper chose to include, so it's an association, not a controlled comparison.

7:42Paige: The authors say the same. They also offer a for why. In , the makes about six when it reads the five-forgery folder itself. When it , it makes about three. Their reading is that delegation removes the occasion to verify, not the ability. With the document in hand, checking a date is a tiny motion. With only a summary, there's no page to flip back to.

8:07Eric: And they label that a . They describe the experiment that would test it, and then write, "We have not run it." There's one hint in its favor. In a code , where verification is a quick search away, they report the attack didn't .

8:24Paige: Now for the complications, and I know you have a list.

8:27Eric: I do. My favorite finding is that they also tried classic , where the planted text gives the reader an explicit instruction. The resisted it everywhere they tested it.

8:39Paige: Their line is, "What succeeds makes no demand. It looks like a record." Think of an accounts department, that deletes every email shouting "wire money now," and then pays the fake invoice.

8:51Eric: But the forgeries were built to be catchable by a careful reader, and missable by a careless one. The authors state that outright, and they map the edges. Remove both flaws, and falls for it 175 times in 180 when reading the folder directly. With , it falls for it 173 times in 180. There's no gap, because there was nothing to catch.

9:14Paige: And where an already fails alone, has nothing to break. In their words, the break lives in one window: a forgery a careful reader catches, in the hands of one who does not.

9:26Eric: So the depends on how much real forged content sits in that window, and nobody has measured that. And there's the claim in the title. For the archivist forgery, it holds as a failure rate. The combined systems land within about one to eight of the cheap model, working alone. But on the five-forgery version, the cheap model alone failed 136 times out of 180. The combined system failed 75 times out of 180. The is rescuing some of those errors. The system also failed on facts the cheap reader handled fine alone.

10:01Paige: So it inherits something like the weak reader's , not its specific errors. And the authors add a noise warning. Rerunning the same fact gave a different outcome about one time in eight. One versus seventy-five survives that easily, and the small gaps don't.

10:17Eric: The delivery route matters too. An identical forged report was adopted 85 percent of the time, when it arrived as a 's . When it arrived as a message from a , the report was adopted 46 percent of the time. The samples are modest, but my read is that models treat their own helper like an instrument, and a colleague like an opinion.

10:40Paige: There's one more experiment, run in simulation and not in the products. An attacker fools a worker, and the worker reports its conclusion to an , which answers from that report and never sees the source. With an honest worker, five aggregators made no errors across 780 trials. Once the worker was fooled, they repeated its error in 751 of 757 cases.

11:02Eric: I'm less impressed by that than the number invites. If one report is your only input, agreeing with it isn't a shock.

11:10Paige: Agreed, and the informative half is the mirror image. Hand the the source record, and errors drop to zero in 780, while the workers stay just as fooled.

11:20Eric: That half I like. The judgment was there the whole time, and the evidence wasn't.

11:26Paige: And in the real , they ran a small defense test with 40 trials per condition. The was 15 failures. Adding a , which makes the helper quote the original record, brought failures down to 3. A that was merely skeptical, told only that the store might be unreliable, brought failures down to 1.

11:47Eric: So a did about as well as a one. At forty trials I'd call that promising, not settled. Still, cheap fixes in a security paper are a pleasant surprise.

11:58Paige: So this is what changes for me. The cheap helper isn't only a cost decision, it's a . The authors point out that the attacker needs no privileged access, only the ability to write a file where an will read it. And they argue that defenses which tag on entry miss this channel, because the 's report is the system's own voice.

12:21Eric: And testing the alone would never reveal it. The authors note that vendors weren't notified. They say the main risk they see isn't misuse. It's overconfidence that a strong model on top makes up for a weak one underneath.

12:36Paige: Here are three takeaways. The first is that on the same facts, strong failed twice in 1,080 direct trials. They failed far more often once a cheaper did the reading.

12:48Eric: The second is that the controls point at the reader's weakness, not at . A clean store or an equal-strength helper produced almost no harm. And all of that is shown for deliberately catchable forgeries, scored on fictional names.

13:03Paige: The third is that contact with the original evidence is what protected these models. Restoring it, or adding a second that doubts the store, brought failures close to zero in small tests.

13:15Eric: So, does the system inherit the strong model's judgment? It does only on the evidence that model reads for itself. On everything else, you're getting something much closer to the helper's.

13:27Paige: If you want the annotated version of this episode, it's at paperdive.ai, with the full transcript. You can tap any technical term for a definition, and find related papers linked by theme. We break down a major AI paper every day, so subscribe and tomorrow's will be in your feed.

13:45Eric: Quick housekeeping. The script was written by 's , and then refined by 's . Paige and I are AI voices from , and we're not affiliated with any of those companies. The paper is "Only as Safe as the Weakest Agent," by Keegan Wang and Anantika Mannby, posted November 12th, 2026.