Definition
Plain language
A training objective that pays extra attention to the examples a model is still getting wrong.
As stated in the literature
A modification of cross-entropy that downweights easy examples and upweights hard ones via a focusing factor; originally from object detection, applied as a cross-domain substitution in Autoresearch agentic training.
Why it matters: It rescues training in imbalanced settings where ordinary cross-entropy gets dominated by easy negatives and the model never learns the hard cases.
For example, when training a tumor detector where 99% of image patches are obviously healthy, focal loss tells the model to stop wasting effort on those and focus on the ambiguous ones.
Heard on the show
“It swaps in focal loss instead.”Episode 053 — An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script