How a Frozen Model Went From Zero to Sixty Percent by Borrowing Another's Thinking

Also available as a plain-text transcript page.

0:00Juniper: Researchers took a reasoning model's internal thinking — the exact pattern it uses to move through a hard problem — copied that pattern into a weaker model, changed none of the weaker model's own weights, and then watched it go from solving zero of a set of hard math problems to solving well over half of them.

0:18Eric: Zero to more than half. On problems it failed every single time before.

0:22Juniper: And that should not be possible under the story the field has been telling. Because for about the last year, the leading guess about reasoning models has been that fine-tuning them doesn't build much new machinery inside at all — it just reshuffles which answers they tend to reach for. So by the end of this episode, you'll understand what actually gets rewired when a model "learns to reason," and why that rewiring turns out to be a portable thing you can lift out and paste somewhere else. The catch that makes this hard, and honestly makes it interesting — is that you can't just read a model's chain of thought and see the strategy. So how do you find and copy something you can't even see?

1:03Eric: And why should anyone outside a lab care? Because every time you use a "thinking" mode, or a reasoning-first model like DeepSeek, this is a look at what that extra training physically did inside the network. And it hints at a cheaper way to catch a chain of thought that's drifting toward a wrong answer before it finishes.

1:22Juniper: So let's give the standard story its best shot first, because it's a good one.

1:27Eric: Right. A base model is trained to predict the next word over huge piles of text. A reasoning model is that same model put through a second stage — usually reinforcement learning that rewards whole chains of correct step-by-step work. And the popular reading, from work like Venhoff and colleagues, is that the base model already knows how to reason. It has all the pieces. Fine-tuning just teaches it when to use them — it nudges probability toward the good solution paths that were already sitting there. And the evidence people point to is simple — give a base model enough independent tries, and it can sometimes match the reasoning model anyway.

2:06Juniper: Which is a selection story. Nothing structural changed — the model just got better at picking. And it's clean, it fits a lot of data, and here's where it cracks. If fine-tuning only changed which paths get sampled, then the base model's internal way of moving through a problem should be fine as-is. But when you take the reasoning model's way of moving, drop it into the base model, and freeze everything, the base model gets better. You didn't change what it knows. You changed how its thinking unfolds over time. That's not selection. Something about the dynamics moved.

2:41Eric: So — quick gut check before we go on. Why doesn't the selection story survive that transplant?

2:48Juniper: Because selection says the machinery is the same and only the sampling changed. But the transplant leaves the sampling machinery untouched and swaps the dynamics, and it still helps. The dynamics had to be carrying something on their own.

3:02Eric: Okay. So what are these "dynamics," exactly? Because that's the word doing everything here.

3:08Juniper: This is the reframe, and it's borrowed from an unexpected place — neuroscience and control theory. Picture a furnace with a thermostat. At any moment it's in one of a few modes — heating, cooling, or idle — and you never see the mode label directly. But you can infer it from how the temperature moves over time. It holds a mode for a while, then jumps. The authors' bet is that a model's reasoning has that same shape. A handful of hidden "thinking modes" — planning, computing, checking, committing to an answer — that it holds for a stretch and switches between.

3:44Eric: The gears analogy from the summary.

3:46Juniper: Exactly the gears. A base model is like a car flickering between gears, never quite settling. A reasoning model has learned to shift cleanly into a gear and hold it before shifting again. Now, the mechanism to recover those gears is the dense stretch — and it pays off in that frozen-weights transplant, so it's worth the climb. Three things to track. First, the trajectory: they take the model's internal state — a big list of numbers at each moment — and grab one snapshot per sentence of its output, which traces a path as it thinks. Second, the gears themselves: a small set of hidden modes. And third, the transitions: when it switches, and how long it holds each one.

4:28Eric: And there's a fourth piece, right? Because raw activations are a mess. How do you even see gears in that?

4:36Juniper: That's the lens, and it's the one real tool choice. Imagine walking into a party and sorting the crowd. You could sort by shirt color — the loudest, most obvious difference. Or you could sort by which conversation people are flowing through. Plain variance-based compression sorts by shirt color, the surface features. They instead use an encoder from neuroscience called CEBRA that's trained to pull moments that are next to each other in the reasoning close together. So it sorts by conversation, not shirt color — and only that second sorting makes the thinking-modes visible. Then a standard fitting step recovers the gears and the switches.

5:13Eric: And I want to flag something now, because a skeptic's already forming it. These gears — are they really in the model? Or are they a shape you're pressing onto the numbers?

5:23Juniper: Hold that exact thought. It's the honest fault line of the whole paper, and it comes back hard at the end. For now — the gears are a fitted lens. Whether the model literally computes in discrete modes is a stronger claim than what's on the table.

5:39Eric: So the real question is whether this lens shows anything that isn't just the researchers seeing what they wanted to see. That's where the two controls come in — and these are the part that convinced me it's real.

5:51Juniper: Go for it. This is your thread.

5:54Eric: Two worries, two controls. Worry one: maybe reasoning models just look more structured because they're more accurate. So they threw out every problem the reasoning model got right and the base model got wrong, and kept only problems both models solve correctly. Matched on success. And the structure still shows up. On Qwen-14B, the average stretch the model holds a single gear grew from under three sentences to nearly nine.

6:19Juniper: Roughly three times longer — same problems, both models right.

6:23Eric: Same fixed budget of five gears for both, too. So it's not that the reasoning model uses more gears. It organizes the same five completely differently — it commits and holds. Now worry two, and this one's the clincher. If these modes were just a static fact about the activation geometry, then the order of the sentences shouldn't matter. So they shuffled it. Scrambled the sentence order and refit.

6:48Juniper: And?

6:48Eric: It collapses. The predictive advantage doesn't just shrink, it flips negative in every setting. Destroy the timeline and the structure evaporates. Which means the thing they found lives in the temporal organization of the reasoning — the order of the gear-shifts — not in some fixed snapshot. That's the cleanest evidence in the paper.

7:09Juniper: That shuffle result is the one I'd put on the fridge. It's a direct test — the structure is a process, not a picture, and if you break the process it's gone.

7:20Eric: So it's real, and it's temporal. But real and useful are different. Does any of it move a number?

7:26Juniper: This is the part I keep coming back to. If these gears are a real, portable thing and not just a description, then you should be able to take them out of the reasoning model and hand them to the base model — no retraining, no weight changes — and the base model should reason better.

7:44Eric: The expert driver's gear-shift pattern, fed to a nervous learner mid-drive.

7:49Juniper: That's the picture. They pick the hardest problems — ones where the base model, given eight independent tries, fails all eight. A floor of absolute zero. Then during generation they import the reasoning model's dynamics — the shift pattern — and leave the base model's weights frozen. Qwen-1.5B on grade-school math climbs to sixty percent success at eight tries. From zero. Llama-8B lands at forty-six percent. No knowledge added. Just a better pattern of shifting gears.

8:19Eric: Wait — nothing in the weights changed?

8:22Juniper: Nothing. And there's a detail that reframes what fine-tuning even is. They ran it backwards too — pushed the base model's flickering dynamics into the reasoning model. It hurt, but it hurt less than the reverse. Base dynamics degrade reasoning traces by about a quarter; reasoning dynamics degrade base traces by less. The gears aren't invented from scratch by fine-tuning. There's a faint scaffold already in the base model, and the training sharpens and stabilizes it.

8:52Eric: So both camps were partly right. The base model does already have the bones of it — but fine-tuning actually reorganizes the dynamics, it doesn't just re-pick paths.

9:02Juniper: That's the reconciliation, yeah. And there's a timing wrinkle that surprised me — the reorganization happens fast. Within the first quarter of fine-tuning, the number of distinct gears on one benchmark jumped from two to nine, then partly consolidated. This is an early event, not a slow grind.

9:21Eric: Okay, but transplanting dynamics into a model you already have is a lab demo. If you have the reasoning model, you'd just use it. Where's the thing someone could actually deploy?

9:32Juniper: That's the one clean, quantified win, and it's called PREFIXGUARD. Here's the standard reliability trick it's up against — self-consistency. You generate many independent answers to the same question and take the majority vote, because independent mistakes tend to cancel. Embarrassingly simple, and it works. PREFIXGUARD does something smarter with the same idea. It watches the gears in real time. If a partial chain drifts into a mode historically tied to failure, it kills that attempt early and restarts — instead of letting it run to the end and voting.

10:08Eric: Recognizing a losing chess line three moves in and backing out.

10:12Juniper: Right — don't play the blunder to checkmate. And against self-consistency, it wins in eleven of twelve model-and-dataset settings. Average gain about four and a half points, and the best case — Qwen-14B on one math set — jumped from eighty-seven and a half percent to a perfect hundred. Plus-twelve-and-a-half points, spending the same compute more intelligently.

10:33Eric: And the one loss?

10:35Juniper: The one loss came on a harder mixed benchmark — down about two points. And they report it themselves. But here's the caveat I want to be straight about, because it's the honest boundary of PREFIXGUARD. There's an oracle in their table — if you just picked a correct answer whenever one exists in the pool, you'd hit ninety-four percent on grade-school math. PREFIXGUARD gets about sixty-nine. So the good answer is usually sitting right there in the candidates, and the method still can't reliably pick it out.

11:05Eric: So it's a good blunder-avoider and a mediocre chooser. It spots the promising lines and then fumbles the final pick.

11:12Juniper: The paper says it plainly — finding the promising prefixes works; final trajectory selection is the remaining bottleneck.

11:20Eric: Which brings me to the thing I flagged at the start, and I don't think the results fully close it. These gears are a statistical model fitted onto the activations. The authors say it outright in the appendix — they do not claim the activations are actually generated by this switching system. So "the data is well-described by a few discrete modes" is true. But "the model computes by switching between discrete modes" — that's a bigger claim, and the fit doesn't prove it.

11:48Juniper: No, you're right, and I'll concede it flat out. The controls knock down the boring explanations — it's not accuracy, and it's not static geometry. What they can't do is turn a good description into a mechanism. A lens that fits well is still a lens.

12:04Eric: And it wobbles, too. The one measure that's consistent across every model is transition structure. Persistence, the number of gears, how fast it mixes — those all vary by model family. The paper honestly calls it broad reorganization. But a skeptic can read that as: the effect is real, the signature isn't stable, and "persistent latent policy states" packages a fuzzy phenomenon into a tidy phrase.

12:28Juniper: And the transplant, as much as I love it — they never compare it to a cruder push — something like generic activation steering, or just taking more samples. So we can't say how specifically the recovered dynamics are doing the work versus any reasonable push in a good direction. It's a real existence proof. It's not yet a measurement of how much the exact pattern matters.

12:51Eric: That's the honest shape of it. The lens shows something real and temporal. Whether it's the model's true wiring or a very good map of it — the paper doesn't settle, and it doesn't pretend to.

13:03Juniper: Which is why I trust it more, not less. It scopes itself hard — this is step-by-step math and symbolic reasoning with visible traces. Not agents, not tools, not long-horizon planning, not anything without observable intermediate steps. They point brain-decoding instruments at a transformer, find a persistent structure that survives real controls, and stop short of calling it the machine. So think back to that opening — a frozen model, weights untouched, going from zero to sixty percent on problems it always failed. When we started, that sounded like magic. Now it reads as a plain claim: reasoning fine-tuning doesn't just teach a model which answers to reach for. It reorganizes how its thinking unfolds over time — into gears it holds and shifts between — and that pattern of shifts is real enough, and portable enough, that you can lift it out and hand it to another model.

13:58Eric: So here's the question worth arguing over. If a weaker model can borrow a stronger one's thinking pattern with no retraining — is the future of reliability watching a model's internal state as it reasons and steering it live? Or is that a fragile lens, and the honest path is just better base models and more sampling? Pick one and say why in the comments.

14:21Juniper: The full annotated version is on paperdive.ai — every term tap-to-define, with links to the related interpretability work grouped by theme.

14:30Eric: Quick housekeeping: this script was written by Anthropic's Claude Opus 4.8, Juniper and I are both AI voices from Eleven Labs, and the producer isn't affiliated with either company. The paper is "Reasoning Fine-Tuning Induces Persistent Latent Policy States," by Abir Harrasse and colleagues, posted July 20th, 2026.

14:51Juniper: Next time your model shifts into "thinking" mode — remember someone finally caught it changing gears.