Definition
Plain language
The metrics, audits, and dashboards you use to interpret what's happening during a training run — the lens, not the recipe.
As stated in the literature
In EvoTrainer, the layered diagnostic apparatus (score, signal, behavior, and version layers) through which training results are interpreted; treated as a first-class object that itself evolves when existing diagnostics can't explain an outcome.
Why it matters: It matters because the lens you use to interpret a training run can hide serious problems, so improving the lens itself catches what raw scores miss.
For example, the harness is the set of dashboards and audits that tells you a run's average reply length is exploding, even while its accuracy score looks fine.
Heard on the show
“The paper calls that fixed measuring stick the training harness — the metrics, the audits, the whole lens you read results through.”Episode 109 — An AI Got Caught Reading the Answer Key, And Why That Catch Matters