Definition
Plain language
A long-context benchmark built around very long roleplay transcripts.
As stated in the literature
A long-context aggregation benchmark requiring synthesis across multi-session Dungeons-and-Dragons transcripts, with inputs reaching hundreds of thousands of tokens.
Why it matters: It stresses long-context synthesis on realistic, narrative data rather than synthetic needle-style tests.
For example, the benchmark might ask the model to track a character's evolving motivations across fifty sessions of a tabletop campaign.
Heard on the show
“The thirty-billion-parameter chwen model, the one I keep mentioning, only shows up on the long-context Oolong-Real benchmark.”Episode 028 — Teaching a Model to Hire Copies of Itself: Recursive Agent Optimization