How a Tiny Model Too Weak to Plan Cuts a Bigger Agent's Hallucinations by 80%

Also available as a plain-text transcript page.

0:00Juniper: Here's a result that reads like a typo. A model with about five thousand parameters — too small to plan its way through anything, it solves barely half these tasks on its own — takes a real GPT-4o-mini agent and cuts its hallucinations by eighty percent. Not by being smarter than the big model — by being checkable.

0:20Finn: Quick heads up before we go further — this is an AI-made explainer, both voices included. And that eighty percent is the part that should bother you, Juniper, because the little model isn't winning on intelligence. It's winning on a property almost nobody bothers to optimize for.

0:38Juniper: Right. By the end you'll know why a model too weak to do the job is exactly the right tool for keeping a smarter one honest. And it turns on something most of the hallucination literature walked right past — that inside an agent, a hallucination isn't one mistake.

0:55Finn: It's a mistake that builds more mistakes. Which matters because the next wave of AI agents probably won't be capped by how clever the model is — it'll be capped by whether one early slip quietly corrupts everything the agent does after it.

1:11Juniper: The paper opens with a six-step workflow and a single wrong word. At step three, the agent jots down its picture of the world and marks task three "completed." Except it wasn't — a precondition got dropped when the task list was flattened into text, and the environment still had that task sitting at pending. And an LLM agent has no separate memory of truth. It re-reads its own transcript every step and treats whatever it wrote earlier as established fact. It's a journal you can only plan from — and you never go back to check it. So once "task three: completed" is on the page, it's true forever, as far as the agent is concerned.

1:52Finn: And you can watch the damage travel down the dependency chain. Task five needs task three. The agent, believing three is done, tries to run five — the environment rejects it as invalid. Now it's confused, so it papers over the confusion with three more invented status claims, and the episode just runs out the clock. One false token, three broken actions. That's Figure 3, and it's the whole paper in miniature.

2:19Juniper: It's the spreadsheet failure. One wrong cell, and every formula downstream that points at it quietly produces garbage — and that garbage feeds the next formula. The authors put numbers on it: in their agent baseline, a single false claim survives about two and a half steps before it gets flushed out or the episode dies. And by step ten, the agent's chance of a fresh error on any given step is up around thirty-nine percent.

2:47Finn: So the obvious move is to give the agent a real model of the world to check against. And that's where the paper sets up a choice that turns out to be a trap.

2:57Juniper: There are two ways to answer the question every planner has to answer — if I do this, what happens next? One way: train a small neural network to predict the transition. Feed it the state and an action, it predicts the next state. The nice thing is its errors are measurable — you can score its prediction against what actually happened and get an exact number for how wrong it is.

3:21Finn: The catch being it's a weak planner. On these tasks the trained model tops out in the high fifties to low sixties on success rate — call it 0.57 to 0.63 — while the best LLM agent hits 0.67. It can't reason about a goal it hasn't seen. It systematically under-solves anything that needs real understanding.

3:41Juniper: The other way is what everyone actually does — let the LLM be the world model. It imagines the consequences in language, which is flexible and smart and handles novel goals. But now its errors aren't measurable misses, they're confident fictions, and they go straight into the transcript. So you've got a near-sighted local who can't give you the grand tour but knows exactly which streets connect — versus a fluent tour guide who occasionally invents a landmark and says it with total confidence.

4:15Finn: And those two fail in opposite directions, which is the seed of the whole method. The dumb one is grounded but can't plan. The smart one can plan but lies. So instead of picking —

4:27Juniper: — you staple them together. Keep the LLM as the brain, bolt the little trained model on as a second opinion. That's GILP — Grounded Iterative Language Planning. And the clever part is what the small model is actually asked to do. Each step runs a little loop. First, the backbone — the small trained model — scores every action the agent could take and produces a compressed cheat-sheet: here are the promising moves, here's which nodes I predict each one changes, here's how risky it looks. That cheat-sheet goes into the prompt before the agent drafts anything, so the LLM is grounded before it opens its mouth.

5:09Finn: Then the agent does its thing — picks an action and writes out its own imagined state change, which nodes it thinks just flipped. And now you've got two opinions on the same question: the LLM's list of what changed, and the backbone's list. The consistency gate just measures how much they overlap.

5:29Juniper: And that overlap is one cheap number. Take the two lists, divide the part they agree on by everything either of them mentioned. One means identical, zero means total disagreement. The store-receipt version: two people each list what they bought, and you ask what fraction of the combined list both wrote down. When that number drops below 0.30, the gate fires.

5:54Finn: And firing doesn't mean overriding the LLM. It means handing it a targeted note — these specific nodes are in dispute, take another look — and asking it to revise. That's the spell-checker move. A spell-checker can't write your essay, it doesn't argue with your point. It flags the specific words that look wrong and stays silent the rest of the time.

6:16Juniper: And that's the structural break from most grounding work. The usual approach checks the action after it's produced — filter it, rerank it, have a big verifier judge it. GILP grounds the imagined state before the agent samples, then re-prompts during the same step, but only when the two predictions diverge. There's also a quiet risk gate that drops actions the backbone flags as likely to fail — but the heart of it is this: don't verify every step, decide which steps are worth a second look.

6:47Finn: Before the numbers — and the numbers are good — one flag, Juniper, because it shapes how much we should trust them. The cleanest, fully measured result here is the hallucination drop on real GPT-4o-mini calls. The big success-rate jumps you'll hear come from a behavioral simulator, not from live runs across the board. I'll come back to exactly why that matters. For now: the measured result is the one to hang your hat on.

7:13Juniper: Fair. And there's exactly one piece of math holding this together — worth a minute, because it pays off in a guarantee that the agent's error rate can only ever shrink, even when the little model itself is wrong. They call it the hallucination contraction, and the setup is a checkpoint that every potential mistake has to pass before it becomes permanent. Two things matter: how often the gate catches a bad draft, and how often the fix actually repairs it. Measured here, the gate catches about five in six of the agent's hallucinations, and the re-prompt fixes about nine in ten of the ones it catches.

7:52Finn: So multiply those through, and the bad steps that slip all the way past are the ones the gate misses, plus the ones it catches but can't fix — a small slice. The error rate after the gate is the original rate times one minus that catch-and-fix product. It can go down. It cannot go up.

8:12Juniper: With one assumption worth saying out loud — that fixing an error doesn't spawn a brand-new one. The guard never waves a fresh problem through while turning an old one away. Grant that, and the contraction just falls out. And the point of proving it this way is that it never assumes the backbone is right. It openly lets the little model be imperfect, and still guarantees the agent gets better.

8:39Finn: And the theory makes a prediction you can check. If error contracts step by step, the long-horizon gap should be large — and it is. At step ten, the agent baseline's per-step error is near thirty-nine percent; with GILP it's about sixteen. The compounding just... stops compounding.

8:58Juniper: Now the measured headline, the one Finn flagged. On real GPT-4o-mini calls, the hallucinated-state rate — the fraction of the agent's "this node changed" claims that disagree with what actually happened — drops from about eighteen percent to under four. An eighty percent cut. And the cost is roughly a fifth more LLM calls, because the gate only re-prompts the steps it doubts. That's live data, not simulation.

9:27Finn: The bigger success numbers are where you have to be careful, Juniper. Overall task success going from about 0.67 to 0.84, and on long tasks — past ten steps — from 0.47 to 0.76: those come from a behavioral simulator, calibrated against real runs, not from live agents across the board. Here's the saving grace, though. The authors show the simulator under-predicts GILP's real gains. So when they did run live calls, the true numbers came in better than the simulation said. The simulator is a conservative lower bound.

10:03Juniper: And the shape of it is the thing to hold onto — that's Figure 1. Short tasks, three steps or fewer, the agent's already at ninety-six percent, and GILP barely matters. Then watch the agent-only curve fall off a cliff after about ten steps, down into the high forties, while the GILP curve holds in the mid-seventies. All the value lives in the long tail, exactly where errors have had room to compound.

10:31Finn: Two details that make it feel real. First, the lying is sneaky — open-source Llama-3-8B produces valid JSON describing a physically impossible state on about nine percent of steps, versus under half a percent for GPT-4o-mini. The format is perfect; the meaning is fiction. And second, the gate fires more often on exactly the models that hallucinate more — twenty percent of steps for GPT-4o-mini, up to thirty-two for Llama. The trigger is tracking something real, not firing at random.

11:05Juniper: Which brings us to the finding I'd put on the cover. You'd assume a better backbone gives better grounding — make the little model smarter, catch more lies. So they tried the whole ladder, from a tiny MLP up to a graph transformer.

11:21Finn: So the graph transformer wins, and you pay for the extra accuracy.

11:25Juniper: That's the natural guess. It's wrong. The tiny MLP — which gets transition prediction right about eighty-four percent of the time, and on its own solves barely half the tasks — lifts the hybrid to about 0.77 success. The graph transformer, at ninety-nine percent transition accuracy, gets you to 0.77. The grounding value plateaus almost immediately. The paper's line is that the backbone doesn't need to be a good planner to be a useful error signal.

11:57Finn: And that's the real idea here, bigger than any number. Picture a proofreader who can't write the novel and doesn't follow the plot — but instantly notices when a character who died in chapter two is suddenly talking in chapter nine. They can't do the author's job. They catch exactly the continuity error the author keeps making. So grounding an agent comes down to one cheap, checkable opinion — on the easy question, which things changed — because that's precisely where the LLM lies, and precisely where a lie is catchable. The little model never has to solve the task.

12:37Juniper: It's a clean story. Finn, you've been holding the asterisk all episode — go.

12:42Finn: The headline table — the 0.67-to-0.84, the whole twelve-method comparison — isn't live data. It's a simulator, and that simulator was fit to just five real GPT-4o-mini episodes. The authors say it plainly, and they argue it's conservative, and I believe them. But the breadth of the result rests on a model calibrated from a handful of real runs. And the live validation that does exist is thin in a specific way. The real GPT-4o-mini runs cover twenty tasks per benchmark — and success was a perfect hundred percent in both arms, with and without GILP. The tasks were easy enough that the only thing left to measure was hallucinated content. So the real data backs the eighty-percent hallucination cut, and says essentially nothing about whether GILP makes hard tasks succeed in the wild.

13:35Juniper: That's fair — though the hallucination cut is the measured claim we led with, and it's the one that holds.

13:43Finn: It holds. But two more cracks. The four-model chart where everything converges to the same success rate — only the GPT-4o-mini row is measured; the Claude, Gemini, and Llama numbers are calibrated from published benchmarks, no direct calls. Read it as a hypothesis. And the one test that leaves their own graph-planning sandbox — a knowledge-graph traversal — came back statistically inconclusive. Twelve tasks, no significant difference either way. So the method's reach beyond home turf is genuinely untested. And the backbone needs ground-truth transitions to train, which may not exist in a messy real domain — and a confidently-wrong backbone could actively mislead the agent. They don't study that failure mode.

14:29Juniper: I'll grant all of it. What survives is narrow and real: on a model that lies, a cheap trained checker measurably reduces the lying, and the math says it can't make things worse under a reasonable assumption. Whether that generalizes past these benchmarks — that's the open question, and the paper doesn't get to claim it yet. But step back, because the reframing is the real payload, bigger than the method. For years we scored hallucination one answer at a time — wrong output, caught, done. This paper says that in an agent, a hallucination at step three isn't an error, it's a corruption of the agent's belief about the world, and that corruption manufactures more errors downstream. Once you see it that way, the fix stops being "build a bigger, smarter verifier" and becomes "watch the one cheap, checkable question — what just changed — because that's where the lies are, and where they're catchable."

15:27Finn: Which leaves a real fork for anyone building these things. Do you bolt on a tiny, trained, auditable model to police your big one — and accept that it needs ground-truth data and might sometimes be wrong — or do you bet the right move is making the big model stop lying about the world in the first place? If you've shipped an agent that drifts over long horizons, you already lean one way. Make the case in the comments.

15:54Juniper: If you want to go deeper, the full annotated version is on paperdive.ai — every term tap-to-define, the hallucinated-state and propagation-depth metrics linked out, and the related world-model and step-verifier papers grouped by theme.

16:09Finn: Quick housekeeping: this script was written by Anthropic's Claude Opus 4.8, Juniper and I are AI voices from Eleven Labs, and the producer isn't affiliated with either company. The paper is Grounded Iterative Language Planning, out June 26th, 2026, and we put this together three days later.

16:28Juniper: So here's the shift worth keeping: to make a smart model honest, the answer wasn't a smarter one — it was a cheaper one, watching the single question where the smart one tends to lie. Keep a proofreader on the page who can't write the book but never misses the continuity error. See you in the next one.