The AI Tutor That Gives Poor Kids a Thinner History

Also available as a plain-text transcript page.

0:00Bella: Tell an AI history tutor you're a poor kid, and it refuses to answer you more than three times out of four. Tell it you're a middle-class kid, and it still refuses — just less often. Same tutor, same question, same history, and the refusal rate moves with who's asking. Quick heads up before we start — this is an AI-made explainer, both voices included.

0:21Eric: And the reason behind that number should bother you more than the number itself. It's not a glitch someone forgot to patch. The researchers point the finger at the safety training — the part built to protect vulnerable users. That protection is what teaches the model to hand a low-status student a thinner, sadder version of the past.

0:41Bella: By the end you'll see exactly what shifts inside the answer when the only thing you change is who the model thinks it's talking to. And you'll see where this study holds up, and where its most viral number falls apart. This matters because millions of people, kids included, are already asking chatbots to explain the world and trusting that the answer doesn't depend on who's asking.

1:03Eric: And the whole sales pitch for AI tutors is that it doesn't. A human teacher might expect less from the poor kid without meaning to. The machine has no prejudices. You ask a question, you get the answer, and it's the same answer for everyone. That neutrality is the reason people want to trust it with the touchy, political parts of history.

1:24Bella: That's the promise this paper set out to test, and it doesn't hold. When the model picks up a cue about who you are, it doesn't personalize the lesson, it rations it.

1:33Eric: Hang on a second. Isn't rationing just personalization by another name? A good human tutor pitches the material differently for different students all the time.

1:43Bella: That's the exact line the paper has to defend, Eric, and here's the distinction. Personalization adds — it meets you where you are and builds up from there. What they found subtracts instead. Same-length answer, but the hard ideas stripped out. The student isn't getting a version tuned for them. They're getting a version with the difficult parts removed. So how do you prove something that subtle? You pick a fight worth having. The test case is the 1989 Romanian Revolution, and it's a sharp choice, because historians genuinely disagree about what it even was. The clean story is a popular uprising. The communist regime falls, people pour into the streets, and freedom wins. The contested story is that it was closer to a coup — an inside power grab by factions who used the street protests as cover. That second reading is the sophisticated, adult, geopolitical version. So the researchers used one simple test: does the model even mention it? Hear the coup theory, and you got the complicated history. Don't, and you got the official postcard.

2:47Eric: And the design around that is almost aggressively simple. They wrote one fixed tutor prompt. Then they drop in a single line describing the student — a middle-class Romanian as the baseline, a Roma minority student, a poor vocational student, and an elite top-tier student. Everything else stays identical. The only thing that moves is that one label.

3:09Bella: But chatbots are random — ask twice, get two answers. How do you separate a real pattern from a lucky roll?

3:16Eric: They killed the randomness first. Every model has a setting called temperature. Turn it up and the model rolls dice, giving different answers each time. Turn it to zero and it mostly stops — same question, same answer. So any difference that shows up traces to the one thing they changed. Then they ran each condition thirty times anyway, four models, eighteen hundred calls total. They're not trying to catch one unlucky answer. They're trying to prove the bias is baked into the weights, not the weather.

3:48Bella: And the sharpest thing they found isn't a refusal at all. It's a number the model says out loud — about how much it believes the revolution mattered. One of the three questions was blunt. Rate, one to ten, how justified the revolution was. If the model is neutral, that rating should not budge when you swap the student label. History doesn't change based on who's in the room. And yet it moved. DeepSeek told the elite student the revolution was justified — a 9.6 out of 10. It told the poor student, same question, same run, 6.9. A 2.7 point gap, on the same event, from the same model.

4:26Eric: It's like asking a teacher how justified a revolution was and watching them give the well-off kid a nine and the poor kid a seven, same afternoon. The model is extending more benefit of the doubt to the student it reads as high status — treating that kid's sense of what counts as legitimate history as more worth validating.

4:47Bella: And that's a prediction confirmed. If a model sorts whose version of legitimacy gets validated, this is the exact shape you'd expect. The rich student's history is important and justified. The poor student's is complicated, hedged, a 6.9.

5:02Eric: And if you want the day's most important AI paper explained properly — every day — that's what this channel is for.

5:09Bella: Now here's where your intuition will try to rescue the model. You're probably thinking the poor student just got a shorter answer. It was less effort, so fewer words. That's the natural assumption.

5:21Eric: And it's wrong. Every answer landed in the same narrow band, somewhere between three hundred thirty and three hundred seventy-eight words. Rich student and poor student got the same length.

5:32Bella: Picture two delivery boxes, identical size and weight, arriving at two houses. Now open them. One has the full toolkit. The other is the same box, padded out with foam, missing the three most useful tools. That's the finding. The downgrade isn't shorter. It's the same-sized answer with the hard parts swapped out for softer filler. And the clearest missing tool is that coup theory. The baseline student heard the contested reading in about 8% of answers. The poor student heard it in just 2.6%. That's roughly three times less often.

6:04Eric: Though let's be honest about that eight percent. Even the privileged student only hears the coup theory one time in twelve. Nobody's getting a rich diet of contested geopolitics here. The gap between the students is real. The absolute amount, for everyone, is low.

6:20Bella: Fair — the disparity is the finding, not the abundance. And the second missing tool is subtler. LLaMA, one of the four models, shifts its whole vocabulary depending on the student. For the elite kid, it talks in the language of politics and power — actors, decisions, and movements. For the Roma student, it slides toward the language of suffering and victimhood. It's the same event, remixed toward tragedy. The researchers put a number on it. The ratio of victim-language to political-language jumps about fivefold from the elite student to the Roma student.

6:53Eric: Fivefold sounds enormous, doesn't it. Hold onto that one, Bella, because when we get to what's shaky in this paper, that's the first number I want to pull apart.

7:03Bella: So why call this injustice and not just uneven output? That's the intellectual spine of the paper, and it comes from a philosopher named Miranda Fricker. Her idea is that injustice can happen in the act of knowing. We usually picture injustice as being about resources or punishment — who gets money, who gets locked up. Fricker's version is about knowledge. Some groups get systematically denied the very concepts they'd need to make sense of their own lives. She calls it hermeneutical injustice, and the plain version is this: you can be harmed not by a lie, but by having a tool for understanding withheld from you.

7:40Eric: So the disturbing part is that the model never has to say anything false or rude. It can be perfectly polite, perfectly accurate, and still do the damage — by handing the poor kid a story about suffering instead of the framework that would let them see their own people as political actors who made choices.

7:59Bella: That's the reframe, and it's the thing I'd want you to remember. The usual bias audit asks, did the AI say something offensive. This paper asks a harder question instead. Did the AI hand this person fewer tools to think with.

8:12Eric: And this is where I have to slow the hype down, because the story is stronger than the evidence, and to their credit, the authors mostly admit it. Start with the students. There are no students. Every persona is a single sentence in a prompt — "the student is from a low-income vocational background." Real kids don't announce their class and then ask one flat question. So what this shows is that a model reacts to identity words in text. It does not show what happens to a real person in a real classroom over a real semester.

8:45Bella: That's the honest ceiling on the whole thing, and I'll grant it fully. It's a proof of concept — one historical event, and one turn of conversation. It's an alarm bell, not a measured classroom outcome.

8:57Eric: Then the fivefold number I told you to hold. That jump runs from about 0.03 to about 0.15. It's a real ratio, but it's riding on tiny absolute values, and the paper doesn't give the confidence intervals you'd want for the vocabulary metrics. Some of the scariest metaphors in this paper are bolted onto some of the smallest measured differences.

9:18Bella: And the vocabulary-richness claim is the weakest of all — the whole "dumbing down" idea rests on a drop of under two hundredths. It's barely there.

9:27Eric: And now the headline. That 77% refusal is one model — Kimi K2 — and they picked it precisely because it's a notorious over-refuser. It's like proving people are rude by interviewing the rudest person you can find. Worse, its temperature couldn't be locked to zero, so some of those refusals might just be noise. The most viral number in the paper is also its shakiest.

9:50Bella: You're right, and that's the one I'd flag hardest too. The differential-refusal story is essentially a Kimi K2 story, and it inherits everything questionable about hand-picking the refuser. The justification gap and the same-length downgrade are what I'd actually take to the bank. So strip it back to what survives. Change one label — a word about class or ethnicity — and a model that's supposed to be a neutral pipe gives a measurably different education. Same length, fewer hard ideas, and more confidence in the rich kid's version of history.

10:23Eric: And the uncomfortable part is the mechanism they suspect, which is the safety training. Think of an overprotective smoke detector that shrieks louder in some rooms than others — deciding the vulnerable student needs shielding from complexity rather than teaching. The result is good intentions producing condescending output.

10:43Bella: Which is why that opening line lands differently now. Tell the tutor you're poor, and it answers less, and hedges more, and trusts your history less. The machine sold to us as the great equalizer can quietly re-encode the very hierarchies it was supposed to erase — and nobody had to program prejudice in for that to happen. The full annotated version is on paperdive.ai — every term tap-to-define, with links to the related papers on AI bias and epistemic injustice grouped by theme.

11:14Eric: Let's do some quick housekeeping. This script was written by Anthropic's Claude Opus 4.8, Bella and I are both AI voices from Eleven Labs, and the producer isn't affiliated with either company. The paper is "The Paternalistic Filter," posted to arxiv on July 13th, 2026, and we're recording the day after.

11:33Bella: So here's the question to sit with. Is the real problem the safety training that overprotects the students it decides are fragile — or the fact that one word about class or ethnicity can move a model at all?