Definition
Plain language
A score for how well two rankings agree on which items are higher than which.
As stated in the literature
A rank-correlation statistic measuring the proportion of concordant minus discordant pairs in two orderings.
Also called: Kendall's tau
Why it matters: It's a standard way to test whether a cheap evaluator's rankings line up with an expensive one without caring about exact score values.
For example, if two judges rank ten movies and agree on most pairwise orderings but disagree on a few, Kendall tau quantifies that agreement as a single number between -1 and 1.
Heard on the show
“They use a rank correlation called Kendall tau — and for our purposes the only thing to know is that it's a number between minus one and one, where high positive values mean two rankings line up well.”Episode 035 — Why Frontier Agents Ask for Clarification at Exactly the Wrong Moment