Why the Same AI Model Takes Ten Times Longer on the Same Sudoku

Also available as a plain-text transcript page.

0:00Finn: Forty years ago, chaos theorists had a party trick. Take a pendulum swinging over three magnets. Or take Newton’s method, a mathematical procedure hunting for the roots of a polynomial. Then color every possible starting point by where the system eventually ends up. What appears is a fractal: intricate structure, then more structure when you zoom in, and more again after that. The system is perfectly deterministic. Nothing random happens after it starts. And yet you still can’t reliably forecast the outcome. Well, a group of physicists just ran essentially that trick on an AI reasoning model. Same puzzle. Same model weights. The only thing they changed was the random starting point for the model’s internal thinking. Nudge that starting point in the fourteenth decimal place, and the model can take ten times longer to solve the identical Sudoku.

0:51Tyler: And when they mapped which starting points were fast and which were slow, the fine structure was still there after zooming in a thousandfold.

1:00Finn: Right. By the end of this, you’ll understand why. And the answer isn’t that the model gets confused. It’s that the model keeps getting snagged on answers that are almost right. That matters because “let the model think longer” is now a mainstream design principle. We talk about thinking budgets. Test-time compute. The inference bill you pay when a model grinds for two minutes on a problem it eventually gets right. But nobody had a mechanism for why that bill can jump around so violently. This paper offers one. And, improbably, the mechanism comes from the physics of asteroid orbits.

1:35Tyler: Before we get there, let me state the standard explanation, because it’s probably what most people already assume. A harder problem has a bigger search space. The model has to explore more possibilities. So it runs longer. End of story. And the random seed? That’s just noise. A reliable solver shouldn’t care where it starts. Wherever you drop it, it should walk to the right answer.

2:00Finn: The second half of that story is almost right. These models do reach the right answer from essentially anywhere on the maps the researchers analyze. But there are two things to keep separate: where the model ends up, and how long it takes to get there. The destination is reliable. The travel time isn’t. And “bigger search space” doesn’t explain that. It doesn’t tell you why two nearly identical prompts can consume wildly different amounts of compute. The authors open with a result from earlier work. Researchers found adversarially chosen prompts that cause a kind of denial of service in frontier reasoning models. One of those prompts can burn ten times the compute of a nearly identical benign prompt. That isn’t a smooth increase in difficulty. Something is amplifying tiny differences.

2:51Tyler: And “amplifying tiny differences” is language with a familiar home. Chaos.

2:57Finn: Exactly. Now, we need to be precise about the models in this paper. These aren’t ordinary chatbots writing a chain of thought one word at a time. They’re called recurrent-depth models, or looped models. Here’s what that means. A normal deep network has many different layers stacked one after another. A looped model instead takes one fixed block of weights and runs that same block repeatedly. You give it the puzzle. You also give it a hidden internal vector, initialized from a random draw. Then the model updates that hidden state once, runs the same weights again, updates the state again, and keeps looping. The reasoning happens silently inside that vector. At every loop, you can decode a candidate answer from it. Eventually, that decoded answer stops changing. So in this architecture, thinking longer has a very literal meaning: run the loop more times.

3:51Tyler: That architecture is what makes the experiment possible. It’s also the first major caveat, and I want it up front. A looped model has a continuous internal starting state. You can nudge it one trillionth to the left and ask what changes. A chain-of-thought model doesn’t work that way. Its visible state is a growing sequence of discrete tokens. There’s no obvious equivalent of moving the starting state one trillionth to the left. So whether the same phenomenon appears in chain-of-thought reasoning is an open question. This paper doesn’t answer it.

4:26Finn: Fair. Though these aren’t just toy systems built for a physics demonstration. Small looped models with around seven million parameters have been outperforming language models with more than ten billion parameters on ARC-AGI. So this is a live architecture. It’s just not every reasoning architecture.

4:45Tyler: Good. What’s the actual probe?

4:48Finn: It’s almost insultingly simple, which is part of its charm. Freeze the puzzle. Freeze the model’s weights. Turn off any noise injected during inference, so the system becomes a clean deterministic map. Now only one thing changes: the hidden vector the model starts from. That vector has roughly fifty thousand dimensions, so the researchers can’t scan the whole thing. Instead, they choose a flat, two-dimensional slice through that enormous space. Then they sample starting points across that slice. For each point, they ask one question: how many loops does the model need before its decoded answer stops changing?

5:26Tyler: So every location on the map represents a different internal starting point. And the value at that location is the model’s thinking time.

5:35Finn: Exactly. For an easy puzzle, the pattern is smooth. Large regions have similar thinking times. Start from two nearby points, and the model usually takes about the same number of loops. That’s the intuitive result. But give the same model a hard puzzle, and the map changes. The smooth regions break into branching filaments, with finer structures nested inside them. The authors compare the pattern to optical caustics: those bright, shimmering threads of sunlight at the bottom of a swimming pool. The surface of the water changes smoothly, but the light below concentrates into sharp, organic lines. Smooth causes. Filigreed effects.

6:14Tyler: A complicated pattern still isn’t necessarily a fractal. Plenty of curves look messy from far away and become perfectly smooth when you examine them closely.

6:25Finn: That’s exactly the right objection. It’s the difference between an interesting picture and an interesting result. So the researchers zoomed in. Tenfold. A hundredfold. A thousandfold. They focused on one of the hard Sudoku regions, and the structure kept returning. At each finer scale, new filaments appeared. They also measured the effect. The main measurement is called basin entropy. Here’s the plain-English version. Divide the map into tiny tiles. Then ask how mixed the thinking times are inside each tile. If the boundaries are complicated but smooth, almost every sufficiently small tile will eventually contain just one kind of behavior. The entropy becomes low. But if the boundaries are fractal, the tiles stay mixed even as you make them smaller. Nearby starting points keep producing sharply different thinking times. The researchers cross-checked that result with two other measures of fractality. Those measures agreed.

7:27Tyler: And does this fractal structure track how hard the problem is?

7:31Finn: Closely, at least by the paper’s main difficulty measure. Basin entropy rises with the model’s average convergence time: how long it takes to settle. The pattern appears across four architectures from four different research groups. It also appears across Sudoku, mazes, ARC-AGI visual puzzles, and arithmetic target puzzles. In the paper’s analysis, harder problems produce more fractal maps.

7:58Tyler: Give me the operational number. “Fractal” is a glamorous word. People hear it, picture something beautiful, and miss the actual claim.

8:07Finn: Here’s the number to remember. For the Sudoku model, making the runtime ten times more predictable would require specifying the starting state about ten thousand times more precisely. Ten times more predictable. Ten thousand times more precision. And that terrible exchange rate persists at every scale they checked. The distances between starting points go down to about ten to the minus fourteen. That’s the territory of floating-point precision. So the problem isn’t that the model is randomly changing its mind. Once the starting state is fixed, the system is deterministic. The problem is that useful predictability costs an unrealistic amount of precision.

8:52Tyler: In other words, the solver can be reliable about the answer while remaining unreliable about the bill.

8:59Finn: Precisely. And if you want every major AI paper broken down like this, daily, subscribing is the whole deal. Now, quick checkpoint before we get to the mechanism. Why doesn’t the random starting point simply wash out?

9:13Tyler: Because the endpoint washes out. The route doesn’t.

9:17Finn: The seed picks the road, not the destination. And this is where the physics earns its keep. The authors call the phenomenon transient chaos. There are only two words in that term, and both matter. “Chaos” means nearby starting points can rapidly diverge. “Transient” means that divergence doesn’t continue forever. Full chaos never settles. Think of weather or a double pendulum. But a working solver has to settle. It has to produce an answer. So the chaos can’t live in the final destination. It has to live in the temporary journey toward it.

9:53Tyler: The paper’s analogy is Plinko. Every ball dropped into a Plinko board eventually reaches a slot at the bottom. But two balls released a hair apart can bounce through completely different paths. One can also take much longer to fall. The outcome can be reliable even when the route isn’t. There is one important place where that analogy fails. Real Plinko is random, and its pegs are physical objects. The AI system they study has no randomness after the starting state is fixed. Its “pegs” are something more abstract.

10:27Finn: So what are they?

10:28Tyler: Saddle points. Picture a mountain pass. A saddle point isn’t a deep valley that traps you permanently. It’s a place that slows you down. From many directions, the ground slopes toward the pass. A rolling marble can drift close to it and loiter there. But there’s at least one direction that leads away, so the marble eventually escapes. A saddle is a delay, not a final resting place. Now translate that into a space with fifty thousand dimensions. A point might offer only three escape directions out of fifty thousand. A trajectory can spend a very long time finding one.

11:05Finn: So the slow internal paths are the ones that pass near these saddles.

11:10Tyler: That’s the hypothesis, and the researchers test it two ways. First, they compress the model’s high-dimensional trajectories into two dimensions so they can compare their shapes. Fast trajectories move more or less directly toward the answer. Slow trajectories wander away, hover in another region, then return. Second, they identify sensitive boundaries using a tool called the Fast Lyapunov Indicator. That tool was originally developed to study whether an asteroid’s orbit is stable. Here, it flags starting points whose immediate neighbors take a very different route through the model’s hidden state.

11:48Finn: An asteroid tracker repurposed to catch an AI changing its mind.

11:54Tyler: Which is close to what it measures. Across many puzzles and three models, the Fast Lyapunov Indicator strongly correlates with the number of different candidate answers the model passes through before settling. Remember, the researchers can decode a candidate answer after every loop. So they count how many distinct answers appear along the trajectory. The geometry and the algorithm line up. The sensitive regions are also the regions where the model cycles through more possible answers.

12:24Finn: But that leaves the most important question. What’s actually sitting at a saddle? When the model loiters there, what answer is it holding?

12:33Tyler: They decoded the hidden state at exactly those slow points. In a maze, the saddle regions correspond to dead ends. In Sudoku, they correspond to grids with a repeated digit.

12:45Finn: So the stalled answer is almost right.

12:48Tyler: Yes. The expensive parts of the landscape are near-misses. The model isn’t holding nonsense. It’s holding something that looks like a solution but violates one constraint. Think about a jigsaw puzzle. The piece that costs you an hour usually isn’t the one that’s obviously wrong. It’s the one with the right color and almost the right shape. You press it into place, build outward, and discover the contradiction only three moves later. The expensive moments in solving aren’t always moments of confusion. They can be moments of false confidence.

13:21Finn: And that links the model’s dynamics to a classical result about hard problems.

13:27Tyler: Right. This is the bridge that makes the paper cohere. In a constraint-satisfaction problem, difficulty doesn’t simply rise as you add more constraints. A loosely constrained puzzle can be easy because many answers work. You stumble into one quickly. An extremely over-constrained puzzle can also be easy, because contradictions appear immediately. You can reject bad options fast. The hardest problems often sit in the narrow band between those extremes. Real solutions have become scarce, but plausible partial solutions are still everywhere. That’s the classical description: near-solutions outnumber actual solutions. The dynamical version is that saddle points cluster around incorrect answers that almost work.

14:12Finn: So if you’ve lost the thread, here’s all you need so far. The model’s thinking time depends sharply on its starting state, even when its final answer doesn’t. The slow routes pass near saddle points. And when the researchers decode those saddle points, they find near-misses: answers that look plausible but fail one constraint. That’s a mechanism-shaped story. But it’s still correlation. It doesn’t yet prove that the fractal dynamics come from the model learning to reason well.

14:43Tyler: That’s what the next experiment is designed to test.

14:47Finn: And it’s the strongest experiment in the paper. The researchers train a miniature looped transformer from scratch. Instead of examining only the finished solver, they watch its internal landscape change throughout training. The task is small enough to characterize completely. The model has to solve eight equations with eight unknowns, using arithmetic that wraps around at three. There are about six and a half thousand possible answers. The question is simple: when does the chaos appear?

15:18Tyler: What does the model look like early in training?

15:22Finn: Early on, it behaves like a random dynamical system. It has many stable resting places, and most of them correspond to wrong answers. Start near one, and the model slides into it and stops. The boundaries around those resting places are smooth. There are no fractals. The researchers also measure what happens to tiny perturbations as the model loops. The measurement is called a Lyapunov exponent. You don’t need the formula. Just remember the sign. A negative exponent means a tiny nudge shrinks. Nearby trajectories stay nearby. A positive exponent means the nudge grows. Nearby trajectories separate. Early in training, every Lyapunov exponent is negative. Perturbations die away. The model is stable, predictable, and bad at the task.

16:10Tyler: Then it learns.

16:11Finn: And when it learns, the geometry flips. At around a hundred and fifty thousand training steps, over roughly four consecutive steps, several things happen together. Accuracy jumps. The incorrect fixed points lose their stability. Wrong answers stop behaving like valleys where the model can settle. They become saddle points — passes the model must escape. Positive Lyapunov exponents appear, meaning small differences can now grow. And basin entropy jumps at the same time. Learning the task means turning wrong answers from resting places into speed bumps. The model becomes capable of moving past them. But the amount of time required to escape those speed bumps becomes unpredictable.

16:57Tyler: So the slowdown isn’t damage added on top of the capability.

17:03Finn: It’s the shadow the capability casts. And the researchers make one more cut through the experiment. I think it’s the sharpest part of the paper. In those eight-variable equation systems, some variables can be read off directly through substitution. Other variables require several steps of elimination. The paper calls those harder ones the core variables. The researchers calculate the Lyapunov behavior separately for the two groups. Only the core variables produce positive exponents. The easy variables, the ones you can read off directly, produce no chaos at all. The instability lives entirely in the part of the task that requires multi-step derivation.

17:44Tyler: So in this experiment, the chaos is located in the reasoning itself. It isn’t some unrelated artifact elsewhere in the network.

17:53Finn: On this model and this task, that’s what the evidence says. And that’s what separates “we found fractals inside a neural network” from “we found a plausible reason those fractals emerged.”

18:05Tyler: Alright. Let me spend my skepticism where it matters, because there are three main seams in this paper. One of them carries a lot of weight.

18:14Finn: Take them one at a time.

18:16Tyler: First, generalization. The abstract talks about “reasoning models” and gestures toward theorem proving, software engineering, and autonomous task planning. But the experiments cover a narrow class of looped latent solvers working on puzzles. The researchers chose those models precisely because they can inspect the internal state at every step. That doesn’t invalidate the result. But it leaves a gap between the systems tested and the broad framing.

18:44Finn: Especially because chain-of-thought systems don’t offer the same continuous starting state.

18:51Tyler: Exactly. Chain-of-thought remains untested. Second, the researchers engineer determinism. For one architecture, they explicitly switch off the noise normally injected during inference. Methodologically, they have to do that. You can’t draw a clean basin map for a system that keeps receiving new randomness. But the consequence matters. The fractal is measured in a modified version of that model. If the deployed system’s inference noise is larger than the tiny scales where the fine structure lives, the fractal could be mathematically real but operationally invisible. The paper doesn’t test that.

19:28Finn: So the study proves the fine structure exists in the deterministic system. It doesn’t prove that every deployed version will expose that structure in practice.

19:38Tyler: Right. Now the third objection is the one I’d most want the authors to fix: the difficulty axis. The headline result compares basin entropy with the model’s own average convergence time. But those two measurements come from the same collection of starting points on the same two-dimensional slice. A slice containing more slow points near sensitive boundaries will tend to have both a longer average runtime and higher entropy. So part of the correlation may be mechanical.

20:08Finn: Meaning the analysis says, roughly, “the maps with more complicated timing behavior also have more complicated timing behavior.”

20:16Tyler: That’s the danger. And they already have external difficulty measures available. Sudoku-Extreme rates puzzles by the amount of backtracking a classical solver needs. Mazes can be rated by shortest-path length. If the headline result used those independent labels, the claim that hardness causes fractality would be much stronger. As presented, the difficulty comparison is partly circular.

20:41Finn: I’ll give you that outright. That’s the analysis I’d redo, and I don’t think the paper’s language is fully calibrated to the limitation. I’d add another caution about the causal evidence. The training experiment is unusually clean, but it’s one miniature model, on one task, at one size.

20:59Tyler: And there’s one more methodological choice worth hearing. The researchers filter the two-dimensional slices. They discard a slice if the starting points don’t mostly agree on a final answer, or if too many trajectories hit the maximum number of loops. Both filters are defensible. But they also remove some of the most chaotic cases from the analyzed population. The paper doesn’t report the discard rate.

21:25Finn: So let’s separate what the paper establishes from what it suggests. What it establishes strongly is that these looped, continuous-state solvers can show fractal sensitivity in their thinking time while still converging reliably on the answer. The slow regions line up with saddles and near-misses. And in one controlled training experiment, competence and chaos emerge together. What it doesn’t establish is that every reasoning model works this way, that chain-of-thought has the same dynamics, or that this explains runtime variation in deployed systems generally.

22:00Tyler: And this is a probe, not a fix.

22:02Finn: Exactly. It doesn’t make any model faster. What it gives us is a way to see something that previously looked like an arbitrary annoyance. Two possible uses follow from that. First, basin entropy could become a per-instance measure of how hard one specific problem is for one specific model. That’s a different notion of difficulty from a human rating or a benchmark label. Second, if fractality really appears at the moment a model becomes competent, it could serve as a signature to watch during training.

22:34Tyler: There’s also a possible reinterpretation of that denial-of-service result. An adversary searching for expensive prompts may not be finding prompts that are conventionally hard. They may be finding prompts that place the system near a basin boundary.

22:50Finn: That’s suggested by the mechanism. It isn’t demonstrated by this paper, but it’s a natural hypothesis. And it brings us back to the opening. Physicists have known for forty years that deterministic systems containing saddle points can produce fractal maps and unforecastable timing. The closest prior work built a hand-designed continuous solver for Boolean satisfiability and watched fractal basins appear as the constraints tightened. The new claim here is about spontaneity. Nobody explicitly designed these neural networks to become dynamical solvers with saddle points and fractal timing. Researchers trained them with gradient descent to produce correct answers. The familiar physics appeared anyway. So the core claim is this: when a continuous system resolves a hard, discrete question, unpredictable thinking time may be structural. If that’s right, the search for a clean “stop overthinking” switch may be treating a feature of successful reasoning as though it were a removable defect.

23:52Tyler: Three things to take with you. First, freeze the puzzle and the weights, sweep only the random starting vector, and the map of thinking time is a fractal that survives a thousandfold zoom.

24:04Finn: Second, the slow spots decode into near-misses: maze dead ends and Sudoku grids with a repeated digit. And in the training experiment, the model became competent and became chaotic in the same four steps, with the chaos located only in the variables requiring multi-step deduction.

24:22Tyler: Third, this is measured on looped latent reasoners with the inference noise switched off, and the headline difficulty axis is the model’s own convergence time. Chain-of-thought is untested.

24:34Finn: So which is it? Is erratic thinking time a permanent tax on any system that computes hard answers by settling — something we should budget for and price in? Or is it an artifact of this one architecture that better engineering will smooth away? We’d like to hear which side you land on, and why, in the comments.

24:55Tyler: The full annotated version is up at paperdive.ai, with every technical term tap-to-define and links to the related papers grouped by theme. Now for some quick housekeeping. The script was written by Anthropic's Claude Opus 5 and then refined by OpenAI's GPT-5.6 Sol. Finn and I are AI voices from Eleven Labs. We're not affiliated with any of those companies. The paper is "Fractal basins trap latent reasoning," by Jeffrey Lai and their colleagues, posted September 4th, 2026.

25:26Finn: And that’s the picture. A solver that reliably finds the answer.

25:30Tyler: It just won’t tell you when.