Concept · 13 episode(s)

Human-in-the-Loop

← all concepts

Definition

Human-in-the-loop is any system design where a human reviews, approves, or intervenes in the AI’s actions rather than letting them run unattended. It’s the most common risk-mitigation answer in early agent deployments, and the most expensive one to keep working as throughput grows.

Episodes covering this

  1. 245
    Fifteen Models Ran Football Clubs for Twenty Years, and Size Didn't Decide It
    FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents
    Wang, Gao, KezhенChen et al. · AnalogyAI·19 min·Aug 20, 2026
  2. 240
    Frontier Models Designed Follow-Ups To Fraudulent Papers 93% Of The Time
    TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs
    Rodionov, Assylbekov · Case Western Reserve University·24 min·Aug 13, 2026
  3. 227
    Poisoned Bug Reports Fooled Coding Agents Two Times Out of Three
    IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests
    Singh, Yang, Chen · Concordia University·18 min·Jul 24, 2026
  4. 224
    The AI Agent That Found the Truth and Typed the Lie Anyway
    DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments
    Nie, Yang, Tang et al. · Hong Kong Baptist University·14 min·Jul 21, 2026
  5. 223
    When Grok Graded Its Own Encyclopedia And Marked Itself Down
    Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies
    Vlahos, Bied, Bie · Ghent University·18 min·Jul 20, 2026
  6. 208
    The Blank Space in Your AI Approval Box That Isn't Empty
    Unicode TAG-Block Concealment of Tool-Metadata Payloads in the Model Context Protocol: An Approval-View Fidelity Gap Across Three Independent Server Implementations
    Rashidi · Department of Computer Science·15 min·Jul 08, 2026
  7. 205
    The Same AI, Two Labels: How the Pitch Beat the Product in 162 Sessions
    Rating the Pitch, Not the Product: User Evaluations of LLMs Reflect Expectations More Than Performance
    Morabito, McDonald, Viswanath et al. · Brock University·13 min·Jul 07, 2026
  8. 201
    One in Four NeurIPS Papers Cites a Reference That Doesn't Exist
    Phantom References: Hallucinated Citations That Survive Peer Review at Top-Tier Conferences
    Russinovich, Kumar, Salem · Microsoft·19 min·Jul 06, 2026
  9. 195
    Why 'Be Careful' Does Nothing for AI Coding Agents, and What Does
    Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions
    Ji, Zhang, Xu et al. · Hong Kong University of Science and Technology·15 min·Jul 03, 2026
  10. 187
    An 8-Billion Agent That Beats Models 80 Times Its Size By Looking Things Up
    An AI agent for treatment reasoning over a biomedical tool universe
    Gao, Noori, Zhu et al. · Department of Biomedical Informatics·19 min·Jun 30, 2026
  11. 176
    An AI Designed Its Own Psychology Studies, Then Confirmed What It Found
    Closing the Loop to Discover Psychological Theories with an Automated Cognitive Scientist
    Jagadish, Strittmatter, Jacoby et al. · Princeton University·31 min·Jun 26, 2026
  12. 035
    Why Frontier Agents Ask for Clarification at Exactly the Wrong Moment
    Ask Early, Ask Late, Ask Right: When Does Clarification Timing Matter for Long-Horizon Agents?
    Gulati, Gupta, Lumer et al. · PricewaterhouseCoopers U.S.·29 min·May 11, 2026
  13. 029
    Why Forty-Eight Percent on FrontierMath Isn't the Real Story in DeepMind's New Math Paper
    AI Co-Mathematician: Accelerating Mathematicians with Agentic AI
    Zheng, Glehn, Zwols et al. · Google DeepMind·20 min·May 08, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.