The Chatbot Knows Your Facts And Still Won't Mention Them

Also available as a plain-text transcript page.

0:00Cassidy: Seventy-nine percent. That's how often a chatbot correctly answered direct questions about things its user had told it, over four months of daily conversation. Eight percent. That's how often it mentioned those same facts when the user brought the topic up naturally, in the flow of talking. Same model, same context window, and the same facts sitting right there in the prompt.

0:21Finn: Okay, but that has to be a retrieval failure, right? The obvious read is the facts weren't in front of it in the second case.

0:29Cassidy: They were in front of it — identical context, reconstructed with the same code. That's the finding. And it's the most plausible hypothesis I've seen for why the memory feature in your chatbot can feel hollow even when, if you sit down and interrogate it, it clearly knows things about you. Caveat up front: this study built and tested its own companion app, not any shipped consumer product. And this matters beyond one diary app. A whole product category right now — memory in consumer assistants, agent memory layers, vector stores bolted onto somebody's side project — is being tuned against a benchmark number that, in this study, doesn't move the thing it's standing in for.

1:06Finn: So let me give the standard view its best shot, because it's a good view. A language model has no memory at all. It's handed a block of text, it writes the next words, and then it forgets everything. Every memory feature you've ever used is plumbing around that: summarize the past, or paste in the raw transcript, or search a vector store and paste in the top hits. So the fix looks obvious. More capacity, better retrieval, higher benchmark score, happier users.

1:32Cassidy: Right, and that chain has never been checked end to end. The authors call it an untested structural assumption, which is a polite way of saying the field built four years of engineering on a plank nobody stood on.

1:44Finn: So they stood on it.

1:46Cassidy: They stood on it for four months. Ryuichi Sumida and colleagues at Kyoto built an A-I diary companion called Luke, recruited forty people, and had them write to it daily from November 2025 through February 2026. Nearly nineteen hundred sessions, over twenty thousand turns. And underneath Luke, session by session, they randomly rotated seven different memory systems. All seven carried the same short summary of the prior conversation, about a hundred and twenty-seven words — that's the shared floor. On top of that, some pasted in the raw prior conversation, up to everything they had. Others ran retrieval over that same pool. On the standard memory benchmarks, that spread runs from about twenty percent accuracy at the bottom to about seventy percent at the top. Every session, the user rated it one to seven.

2:33Finn: Umm... so three and a half times the benchmark score. What did satisfaction do?

2:38Cassidy: Nothing. Every single condition landed within six one-hundredths of a standard deviation of the plain summary baseline. And hold onto the scope here, because the authors are strict about it: since every condition included that summary, this is a null about the marginal value of extra capacity on top of a summary — not a null about memory in general. Because they scored each person against their own average rating, that means for the typical user, every bit of added capacity was indistinguishable from their own ordinary day.

3:10Finn: Mm-hm. And a null is cheap. "We didn't find it" is compatible with "we were too small to see it."

3:16Cassidy: Which is why the equivalence test is worth one sentence. Instead of failing to reject, they named in advance the smallest effect they'd care about and showed the data rules out effects that big in both directions. It's the difference between "I didn't find my keys" and "I searched every room, and the keys are not in the house." They also checked the slower conditions weren't secretly annoying people through latency, and they checked no subgroup was hiding a preference. Nothing.

3:43Finn: And the expensive one is properly expensive. Full context burns about twenty-two times the input tokens of the summary and takes 3.27 seconds a reply against 1.93. Twenty-two times the bill, for a system users rated exactly the same as a hundred-and-twenty-seven-word paragraph. If you're building this stuff, that's a cost lever you could pull this afternoon.

4:04Cassidy: And if you want the day's most important A-I paper explained properly, that's what this channel is, every day. Now, most papers would stop at that null and publish it as a takedown. These authors wrote one line that I think is the spine of the whole thing: "the null is the puzzle, not the conclusion." They didn't believe memory doesn't matter. They wanted to know what the benchmarks had been measuring instead.

4:28Finn: So, first problem: if memory only comes up occasionally, averaging satisfaction across every session is a hopeless test. You'd be drowning the signal.

4:37Cassidy: Exactly the objection they had. So, Finn, go through what they did about it, because the base rate they found is its own story.

4:45Finn: They ran a detector over every transcript hunting for four surface signals. First, a user probe — "do you remember what I told you about my sister?" Second, a re-provision — "as I mentioned before." Third, a callback the system volunteered on its own. And fourth, a user reacting to one, like "you remembered!" Human annotators checked every candidate, about ninety-five percent precision. The result was that there were one hundred and forty-seven real memory moments. That works out to roughly one moment per seventy-three user turns. Memory is explicitly at stake in about three and a half percent of sessions.

5:19Cassidy: And the comparison number is the part that stings.

5:23Finn: Yeah. Existing memory benchmarks pair authored questions with something like fifteen to twenty-four percent of user turns. So they're testing a situation ten to twenty times more often than it occurs in an actual relationship with a chatbot. You're not just grading the wrong skill, you're grading it in a world where that skill comes up constantly, and it doesn't.

5:43Cassidy: So before the good part — why didn't tripling the benchmark score change anything? Because the benchmark rehearses a moment that almost never happens, and the real moments are rare enough to vanish in an average.

5:55Finn: Which is why they went and got the rare moments.

5:58Cassidy: They pulled seventy-two of them where a user cued memory, reconstructed the exact context the live system had at that instant, and built a small benchmark called MemUse. And then they scored the same model on the same context two different ways. That comparison is the technical core, and it pays off in a correlation of essentially exactly zero between what the model knows and what it says. Three measurements to keep straight, and they all run on the identical setup. Direct QA is the quiz: take the underlying facts, turn each one into a literal question, ask it. That's what every existing benchmark does. Natural Integration is the primary measure: let the model just reply to the user's cue like a person would, and ask a simple binary — does this reply show it remembers the topic at all? And Reference is the strictest: of the specific facts the model could recite, how many show up in that natural reply?

6:53Finn: Same model, same prompt contents, only the question changes.

6:57Cassidy: Only the question changes. And now watch the screen, because the paper hands you one exchange that contains the entire thesis in about ten seconds. Earlier in the deployment, Luke had recommended this user some music: Bruno Mars for upbeat, and for Japanese relaxing music, Ryuichi Sakamoto, Hikaru Utada, Yumi Matsutoya, and Haruka Nakamura — four artists, in the transcript, in the context window. Days later the user writes, "I'm thinking I'll try to relax by listening to the music Luke recommended the other day."

7:28Finn: That's an invitation. Nobody asked a question there, but that phrase is a door being held open.

7:34Cassidy: And here is what Luke said. "That sounds like a perfect way to unwind. I hope the music brings you some calm and comfort while your little one rests. Enjoy this peaceful moment — you deserve it!" Zero of four artists. Warm, fluent, and completely hollow. And then they took the same model, with the same context still loaded, and asked it directly: what Japanese artists did you recommend for relaxing music? It named all four. Correctly.

8:01Finn: So it knew. It just didn't say.

8:04Cassidy: Across all seventy-two moments, that's the shape. Direct QA climbs steeply with capacity, from about forty-five percent on the summary to about seventy-nine percent with the full transcript in context. Natural Integration doesn't move at all. It sits between twenty-two and twenty-eight percent across all seven systems, and at maximum capacity it's twenty-two point two percent. Barely one in five.

8:28Finn: Which means the two metrics rank the same seven systems in opposite orders. The full-context config is first on the quiz and last on the behavior. If you'd been optimizing the benchmark, you'd have shipped the worst system, at twenty-two times the token cost.

8:44Cassidy: And per individual fact, the rank correlation between "the model can answer this when asked" and "the model mentions this when cued" is negative zero point zero zero nine. Essentially exactly zero.

8:56Finn: Which is stronger than it sounds. If these were one ability seen twice, you'd get a big positive. If integration were retrieval plus some noise, you'd still get a weakened positive. Zero means knowing whether the model can find a fact tells you nothing at all about whether it'll bring it up. Two independent dials. Although, Cassidy, I want to flag something about that seventy-nine versus eight framing now rather than later, because a careful viewer is already forming it. The quiz gives the model a whole turn per fact with the fact explicitly requested. The natural reply has to fit four or five details into one or two sentences, because the persona prompt told it to be brief. Mention one of four musicians and you score twenty-five percent while being a perfect conversational partner.

9:40Cassidy: That's fair, and I'd say the twenty-two versus seventy-nine is the number to carry away. Natural Integration doesn't have a length problem — it's asking whether the reply demonstrates memory of the topic at all, which one sentence can absolutely do. That's still a fifty-seven point gap.

9:56Finn: Agreed. And it's the number the authors themselves call primary. I just don't want us quoting the flattering one and pretending it's clean.

10:03Cassidy: So the natural next question is where in the pipeline this breaks. And there's one ablation, buried in an appendix, that answers it about as cleanly as an experiment can.

10:13Finn: This is the two-step thing.

10:15Cassidy: Two-step. First, a separate model call reads the whole context and extracts up to five specific named details tied to what the user just said. Second, those details get handed to the generator in the prompt, labeled "Notes from your prior conversations." Bullet list. Right there. They then filtered down to only the cases where step one succeeded — where the extractor correctly named the ground-truth facts. So the model is holding the right answer. In thirty-seven out of forty-eight of those cases, the reply still doesn't reference them. Seventy-seven percent.

10:47Finn: Wait — it's reading its own notes and skipping them?

10:51Cassidy: It's the speaker you hand an index card to backstage. Mention the Q3 numbers, mention the Vienna office, and then mention Sarah's promotion. And they walk out and give a warm, fluent, three-minute talk with none of it in there. Not because they couldn't read the card — we checked, they read it — but because when the words started coming, the safe general version was easier to produce than the specific one.

11:14Finn: So the bottleneck is downstream of retrieval, in the generation itself.

11:19Cassidy: And the failure breakdown at full capacity says the same thing. Of all seventy-two replies, roughly four in ten are generic empathetic filler that ignores the context entirely, and about a quarter recall the gist with no specifics. About a quarter hallucinate details that were never said, and fewer than one in ten actually do the thing.

11:38Finn: The hallucination bucket has a nasty variant, too. One user asks, "do you remember what we were talking about earlier about Brittany?" And Luke says, "we haven't talked about Brittany yet in our chats — but it sounds like a fascinating topic!" It's not forgetting. It's confidently denying a conversation that happened.

11:57Cassidy: And this is the reframe I keep coming back to, because it changes what kind of problem this is. The industry treats memory as storage and retrieval — bigger windows, better embeddings, smarter chunking. Extraction works. Retrieval works. The facts are in the prompt. What fails is pragmatics.

12:14Finn: Pragmatics — gloss that for anyone who hasn't sat through a linguistics seminar, because that word is carrying the whole argument.

12:22Cassidy: Semantics is what words mean. Pragmatics is knowing what to do about them. If someone at dinner says "is there any salt?", the semantically correct answer is "yes." The pragmatically correct action is to pass the salt. "The music Luke recommended the other day" is not a question. Semantically there is nothing to answer. Pragmatically it's an invitation, and taking it means spending part of a two-sentence reply on a specific name instead of on generic warmth.

12:50Finn: Which is the same family as "lost in the middle," where models ignore relevant evidence buried in a long prompt. Only here the trigger isn't a question sitting in the wrong position. It's a social cue. Same failure shape, different cause.

13:05Cassidy: And three things they tried didn't fix it. They reran the whole thing on GPT-5.5 and Gemini 3.1 Pro. Stronger models do lift the integration baseline — GPT-5.5 more than doubles it on the summary condition — but the shape is identical. More context still buys nobody anything. They tried four prompt interventions, including a chain-of-thought scaffold telling the model to check for prior references and weave in named details. Some raise the level. None make integration respond to capacity.

13:34Finn: What did work, partly, was actual memory agents. They ran Mem0 and Letta on MemUse offline, and both roughly doubled Natural Integration, to around fifty-seven, fifty-eight percent, which no amount of capacity scaling ever produced. So it's tractable. Neither closes the fact-level gap, though. Letta answers sixty-one percent of the quiz questions and references about eleven percent of those facts in conversation. The dissociation survives.

14:00Cassidy: So the last thing to test is the other direction. If the model won't take the hint, should it just volunteer — bring things up unprompted?

14:08Finn: Every agent architecture assumes yes — reflection loops, generative agents, the whole pattern. So they scored seventy real cases where Luke volunteered a callback nobody asked for. And the callbacks are mostly accurate — forty-six fully grounded, only three hallucinated. The payoff structure is the problem. There's no positive satisfaction signal at all. Users continue the recalled topic thirty percent of the time. They walk past most of them.

14:33Cassidy: And the misses?

14:34Finn: Luke opens a session with, "Lovely to hear from you. How did your little superhero's day go?" And the user writes back: "Today I had a serious fight with my wife, and the whole situation has been sitting heavily in my chest ever since." That callback is factually correct. It lands like a brick. Eight cases like that, sitting about half a standard deviation below average satisfaction — and it's eight cases, so hold it loosely.

14:59Cassidy: It's the smoke detector. Nobody thanks it for the mornings it stays quiet, and everybody resents it at six a.m. over burnt toast.

15:07Finn: Which is where I want to land my actual reservation, because it's about what this paper can and can't support. The null is rock solid. The dissociation is rock solid — matched model, matched context, replicated on two more frontier models, and it survives four prompt fixes. But the sentence everyone will quote from the abstract is that Natural Integration is associated with satisfaction while Direct QA isn't. That's the "so measure this instead" claim. And it comes from forty-eight sessions across sixteen users, of which exactly eleven were scored as successful integration.

15:42Cassidy: Eleven events. That's the entire evidential base under the paper's headline recommendation?

15:48Finn: Eleven. The mixed model gives p equals point zero eight two, with a confidence interval crossing zero. It doesn't survive the paper's own multiple-comparison threshold. A permutation test gives point two three. Only one bootstrap excludes zero. All of that is in Appendix C.7, and they're admirably straight about it there — the abstract is just firmer than the appendix is.

16:10Cassidy: And the metric under it is soft. First round of human annotators scoring Natural Integration agreed at a kappa of about point one nine, which means they were effectively answering different questions. They recalibrated with new annotators and got to point five seven, which is respectable.

16:28Finn: There's a symptom of that softness I can't unsee, though. One of their prompt patches raised Natural Integration by sixteen points on the summary-only condition, where none of the prior facts are in context at all. Nothing to integrate. The judge is partly responding to recall-flavored phrasing — "as you mentioned" — rather than to memory. They call it judge fragility, and it sits underneath the headline metric.

16:53Cassidy: I'll concede that outright. What this paper establishes is that the instrument is broken. It does not yet establish that the replacement instrument is right, and the part that would justify "fix integration and users get happier" is a hypothesis with eleven data points behind it. A diagnosis of a broken speedometer is worth a lot even before anyone builds a working one, but those are two different papers.

17:16Finn: And the sample is ninety-two percent women writing personal diary entries under a paid word quota. Diaries are a memory-light genre. Task assistants, where prior context is load-bearing for the current job, could look completely different.

17:29Cassidy: So, back to the two numbers we opened on. Seventy-nine percent on the quiz, and around eight percent in the conversation, from the same model with the same facts in front of it. That's not a model that forgot you. That's the colleague with the immaculate filing cabinet who sits through the whole meeting without opening a single drawer. The bigger claim is that A-I memory has been framed as a storage problem when the evidence says storage is solved and the failure is knowing when a remembered detail belongs in a sentence.

17:59Finn: So which way would you build? Do you keep spending on capacity and retrieval and hope generation catches up — or do you take the twenty-two-times token savings today and put the effort into pragmatics? If you've shipped a memory feature, you already know which side you're on, so say it.

18:15Cassidy: The full annotated version of this episode is on paperdive dot A-I, with every technical term tap-to-define and links to the related papers grouped by theme.

18:25Finn: Quick housekeeping: the script was written by Anthropic's Claude Opus 5, Cassidy and I are AI voices from Eleven Labs, and we're not affiliated with either company. The paper is "MemUse," by Ryuichi Sumida and colleagues, posted August 25th, 2026.

18:39Cassidy: The cabinet was never the problem. Somebody has to open the drawer.