Definition
Plain language
The formula that decides how many points a run is worth.
As stated in the literature
The explicit objective used to grade agent behavior; its weights encode designer opinions (e.g. discounting idle cash), so behavioral correlations partly measure agreement with the benchmark's preferences rather than universal competence.
Also called: scoring functions
Why it matters: Because the formula encodes the designers' opinions, a high score partly measures agreement with those opinions rather than skill in any absolute sense.
For example, a benchmark that docks points for holding too much idle cash will rank a cautious saver below a spender, even if saving was reasonable.
Heard on the show
“Or just calling the scoring function on something it knows will pass?”Episode 046 — When the AI Optimizer Edits the Grade Book: Why Harnessing Evolution Needs a Wall