evaluator

Definition

A function that scores how good a candidate solution is and what was wrong with it.

In optimize-anything, a domain-specific function returning both a scalar score and structured side information about failures, replacing the gradient signal used in numerical optimization.

Also called: evaluators

Mentioned in 9 episodes

1. 067
   An AI Just Solved a 1996 Erdős Problem—and the Simplest Agent Won
2. 065
   One Loop to Optimize Them All: A Universal API for LLM-Driven Discovery
3. 059
   Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward
4. 055
   Why LLM Judges Flip Their Verdicts When You Change the Question Format
5. 046
   When the AI Optimizer Edits the Grade Book: Why Harnessing Evolution Needs a Wall
6. 025
   The Missing Gradient Term That Predicts Sycophancy in RLHF
7. 019
   When the Best Reward Model Trains the Worst Policy: Inside EvoLM
8. 012
   Why AI Coding Agents Keep Trying to Debug Without a Debugger
9. 007
   Exploration Hacking: When Models Sabotage Their Own RL Training