Concept · 1 episode(s)

Baseline Comparison

← all concepts

Definition

Baseline comparison is the discipline of measuring a new method against simpler alternatives — a previous SOTA, a trivial heuristic, a well-tuned classical method — before claiming you’ve learned something. A surprising fraction of reported improvements evaporate when someone tunes the baseline as carefully as the new method.

Episodes covering this