Definition
Plain language
A benchmark of medical questions used to test how well models reason about health topics.
As stated in the literature
An evaluation suite of medical question-answering and clinical-reasoning tasks, used in EvoLM-style cross-domain transfer experiments to test whether learned evaluative rubrics generalize to expert health domains.
Why it matters: Medical reasoning has real stakes, and a dedicated benchmark lets researchers see whether general training improvements actually transfer to expert clinical settings.
For example, a HealthBench question might describe a patient's symptoms and lab results and ask which differential diagnoses to prioritize.
Heard on the show
“Then they evaluate the rubrics on HealthBench, which is medical, and on a research question-answering benchmark.”Episode 019 — When the Best Reward Model Trains the Worst Policy: Inside EvoLM