Definition
Plain language
A football-club management video game used as a twenty-year endurance test for AI assistants.
As stated in the literature
A deterministic sixteen-club, twenty-season simulation with ~374 decision points and 26 tools, scored arithmetically by the engine; each decision opens with no chat history, making self-curated memory part of the capability under test.
Why it matters: It tests whether an assistant can stay coherent and purposeful over hundreds of linked decisions, which is much harder than answering one question well.
For example, a model is handed a mid-table club and asked to run it for twenty seasons of transfers, contracts, and budgets, with each decision arriving as a blank conversation.
Heard on the show
“So, it's called FM-Bench, and the thing to hold onto is that it's a deterministic football-management simulation.”Episode 245 — Fifteen Models Ran Football Clubs for Twenty Years, and Size Didn't Decide It