Definition
Plain language
A rating number that ranks players based on who beats whom.
As stated in the literature
A chess-derived rating system that converts pairwise comparison outcomes into scalar scores; used in the DeepMind Erdős evolutionary agent to rank proof sketches via LLM-as-judge tournaments.
Why it matters: It turns noisy pairwise judgments into a stable global ranking, which is useful whenever you need to pick winners but only have relative comparisons.
For example, candidate proof sketches are pitted against each other in pairwise LLM-judge comparisons, and the resulting Elo scores decide which sketches get expanded next.
Heard on the show
“You do enough of those rankings and the system converts them into Elo scores — the same idea behind chess ratings.”Episode 067 — An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won