Glossary · Term

rank correlation

← all terms

Definition

Plain language

A measure of how closely two orderings of the same things agree.

As stated in the literature

Correlation computed on ranks rather than raw values (e.g. Spearman or Kendall tau); used here to test whether an ordering at year five predicts the final year-twenty ordering.

Also called: rank correlations

Why it matters: It answers whether an early standing predicts the final one, which tells you if a long test is really measuring anything the short version missed.

For example, if the top three finishers at the halfway point are still the top three at the end, the two orderings correlate strongly even if the point totals changed a lot.

Heard on the show

“At year five the lines are a tangle, and the rank correlation with the final order is zero point one nine, essentially no signal.”
Episode 245 — Fifteen Models Ran Football Clubs for Twenty Years, and Size Didn't Decide It

Mentioned in 5 episodes

  1. 245
    Fifteen Models Ran Football Clubs for Twenty Years, and Size Didn't Decide It
  2. 069
    When Smarter Models Forecast Worse: The Hidden Failure Mode in LLM Predictions
  3. 058
    Why Upgrading Your AI Auditor to a Smarter Model Can Make Your System Less Safe
  4. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
  5. 035
    Why Frontier Agents Ask for Clarification at Exactly the Wrong Moment

Related terms