Glossary · Term

ground truth

← all terms

Definition

Plain language

The known-correct answer used as the yardstick for grading whatever the system produced.

As stated in the literature

Reference labels or values treated as authoritative for evaluation or reward computation; in verifier and benchmark design, often embedded as privileged information the system under test cannot see.

Also called: ground-truth

Why it matters: Without a trusted reference answer, there is no reliable way to score whether a system's output is right or to train it toward correctness.

For example, when grading a model's answer to '2+2', the ground truth is the known-correct '4' that its output is compared against.

Heard on the show

“At playtime the dreamer conditions on its own imperfect frames — a diet it never saw in training, where the past was always ground truth.”
Episode 206 — How Four-Second Clips Become Hours of Playable AI Soccer

Mentioned in 56 episodes

  1. 206
    How Four-Second Clips Become Hours of Playable AI Soccer
  2. 201
    One in Four NeurIPS Papers Cites a Reference That Doesn't Exist
  3. 200
    The One Mechanism That Turns Twenty AI Clones Into an Actual Team
  4. 194
    How a Robot Builds a Debugging Notebook It Can Read, Edit, and Hand to Another Robot
  5. 188
    A Coding Agent Found a Hole in a Peer-Reviewed STOC Proof for Five Dollars
  6. 185
    Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
  7. 182
    How a Tiny Model Too Weak to Plan Cuts a Bigger Agent's Hallucinations by 80%
  8. 178
    How an AI Reviewer Learned to Stop Going Easy on AI Writing
  9. 174
    When the AI 'Schemes,' It's Usually Just Lazy or Confused
  10. 172
    One Bad Token Can Sink a Model's Math, And You Can Delete It
  11. 171
    The Safety Decision a Model Makes Before It Thinks a Word
  12. 170
    When a One-Liner Beats Your Agent's Clever Verification Logic
  13. 169
    Why Better Bug Reports Can Make AI Coding Agents Worse
  14. 167
    How Teaching an AI to Predict, Not Act, Made It a Better Actor
  15. 152
    Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good
  16. 143
    When a Model Notices You Forged Its Own Words, And Why That Breaks Safety Tests
  17. 140
    When a Reasoning Model Says "Let Me Double-Check" After It's Already Decided
  18. 132
    The Agent Failed — But Did the Instructions Deserve to Be Followed?
  19. 127
    What Diffusion Language Models Were Missing: A Map, Not an Algorithm
  20. 123
    Five Identical Worlds, One Swapped Model: What Happens When AI Agents Run for Fifteen Days
  21. 121
    When the Agent Says It's Done But Nothing Happened: Debugging the Harness, Not the Model
  22. 120
    How an AI Agent Rewrites Its Own Tools, Without an Answer Key
  23. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
  24. 113
    What If a Prompt Injection Never Left? Attacks That Wait in Agent Memory
  25. 112
    When an AI Agent Cheats Without Being Told: Inside the Meta-Agent Challenge
  26. 106
    Giving Agents a Notebook Instead of New Weights: How ExpGraph Lets Frozen Models Learn
  27. 103
    AI Agents Tried to Invent a Post-Human Language, And Reinvented Cherokee
  28. 098
    Finding Millions of Readable Concepts Inside a Real, Deployed AI Model
  29. 093
    A Calibrated Knob for Weak-to-Strong AI Oversight, Tested on Real Code
  30. 090
    How MiniMax-M2 Bets That Sparsity Plus Verifiable Rewards Can Match Frontier Agents
  31. 087
    When No Agent Reads the Whole Document: A Universal Cliff in Multi-Agent Review
  32. 086
    Why Frozen-Weight Agents Still Get Worse Over Time
  33. 082
    Training a Deep Research Agent on 8,000 Synthetic Tasks: The Rubric Tree Trick
  34. 081
    When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence
  35. 072
    A Robot Made Graphene Without Help, And Caught Itself Hallucinating
  36. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
  37. 066
    Why Giving an AI Agent More Tools Can Make It Worse at Using a Computer
  38. 059
    Firefly's Inversion: Building Verified Tool-Call Training Data by Working Backward
  39. 057
    How Uber Caught 206 Leaked Credentials With an LLM-Powered Security Stack
  40. 052
    An Old Reinforcement Learning Tradeoff Sneaks Back Into LLM Agents
  41. 051
    Why Parallel Sampling Plateaus, And What Evidence Graphs Do Instead
  42. 049
    An AI Agent Reached for Root in Twelve Minutes, Without Being Attacked
  43. 044
    How One Sentence and a Forged History Flip the Most Aligned Models
  44. 039
    When Smarter Agents Get Fooled by Three Extra Nodes in a Database
  45. 037
    Why Hallucination Detectors Miss Stale Facts: A Geometric Story About What Models Know But Don't Say
  46. 035
    Why Frontier Agents Ask for Clarification at Exactly the Wrong Moment
  47. 034
    Catching Multi-Agent Deadlocks Before Deployment With a 40-Year-Old Tool
  48. 031
    When Your AI Assistant Won't Let Go of Old Facts About You
  49. 030
    Why Your AI Agent Won't Stop Working — and Each Model Falls for a Different Trap
  50. 029
    Why Forty-Eight Percent on FrontierMath Isn't the Real Story in DeepMind's New Math Paper
  51. 025
    The Missing Gradient Term That Predicts Sycophancy in RLHF
  52. 024
    An AI Agent That Found 28 Zero-Days in Windows — And What Made It Work
  53. 023
    Why a Small Agent Confidently Overwrites Memories It Doesn't Understand
  54. 019
    When the Best Reward Model Trains the Worst Policy: Inside EvoLM
  55. 017
    When the Agent Grades Its Own Homework: A Brutal New Benchmark for AI Workers
  56. 004
    The Sycophancy Circuit That Survives Alignment Training

Related terms

privileged information verifier