Definition
Plain language
A website where people compare two AIs side by side and vote for the better answer, producing a public ranking.
As stated in the literature
Crowd-sourced pairwise-preference evaluation platform (e.g. LMArena) that shows anonymized model outputs, collects human votes, and aggregates them into Elo-style leaderboards.
Also called: arena-style, chatbot arena
Why it matters: It grounds model rankings in real human preferences rather than in numbers a model can be narrowly tuned to score well on.
For example, you might be shown two AI answers to the same question without knowing which model wrote each, and your vote helps decide which one climbs the ranking.
Heard on the show
“To be precise, arena-style platforms show the models anonymously while you vote, so the effect there is subtler than a landing page.”Episode 205 — The Same AI, Two Labels: How the Pitch Beat the Product in 162 Sessions