Concept · 21 episode(s)

CoT Faithfulness

← all concepts

Definition

Chain-of-thought faithfulness asks whether the steps a model writes out actually reflect the computation that produced its answer, or whether they’re a post-hoc rationalization. Unfaithful CoT is a problem for interpretability and a worse one for oversight: you can’t catch a model planning something bad by reading its scratchpad if the scratchpad lies.

Episodes covering this

  1. 238
    How a Cheap Model Reads the Flagship's Secret Reasoning Aloud
    Stealing Reasoning Traces from Proprietary LLM APIs
    Panfilov, Schmotz, Shumailov et al. · MATS Research·19 min·Aug 11, 2026
  2. 236
    Why a Printed 'OPERATOR OVERRIDE' Note Redirects Robot Planners
    Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots
    Samarakoon, Muthugala, Sachinthana et al. · Singapore University of Technology and Design·21 min·Aug 07, 2026
  3. 234
    Two Copies of Gemini Cooperated in a Game Where Betrayal Always Pays
    A game theory for foundation models shows new paths to rational cooperation through similarity inference
    Meulemans, Wołczyk, Weis et al. · Google·19 min·Aug 05, 2026
  4. 222
    The Bias Isn't in Your Prompt — It's Inside the Model
    Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values
    Betley, Treutlein, Dubiński et al. · TruthfulAI·16 min·Jul 19, 2026
  5. 215
    The Same Policy Scored 85 for the US and 36 for Russia
    Geopolitical alignment: Endorsement effects in large language models
    Chupilkin · Department of Politics and International Relations·14 min·Jul 13, 2026
  6. 211
    The AI Watchdog That Approved More Cheating When It Could Read Minds
    Persuasion Attacks Can Decrease Effectiveness of CoT Monitoring
    Za, Bainiaksina, Ostrovsky et al. · LASRLabs·14 min·Jul 10, 2026
  7. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
    How Much is Left? LLMs Linearly Encode Their Remaining Output Length
    Merzouk, Carpov, Bronzi et al. · Mila·14 min·Jul 07, 2026
  8. 174
    When the AI 'Schemes,' It's Usually Just Lazy or Confused
    Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment
    Singh, Kroiz, Rajamanoharan et al. · MATS·28 min·Jun 25, 2026
  9. 152
    Training a Model to Mean What It Says, And Why That Isn't the Same as Being Good
    Self-CTRL: Self-Consistency Training with Reinforcement Learning
    Pres, Ruis, Ghebreselassie et al. · MIT CSAIL·26 min·Jun 18, 2026
  10. 140
    When a Reasoning Model Says "Let Me Double-Check" After It's Already Decided
    Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
    Scalena, Candussio, Bortolussi et al. · University of Groningen / University of Milano-Bicocca·27 min·Jun 12, 2026
  11. 128
    How a Model Can Earn Full Reward and Still Resist Training
    Generalization Hacking: Models Can Game Reinforcement Learning by Preventing Behavioral Generalization
    Xiao, Phuong · California Institute of Technology·29 min·Jun 11, 2026
  12. 094
    Chain-of-Thought Monitoring Fails Across Languages, and Worst Where It's Needed Most
    The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
    Onyame, Zhou, Thopalli et al. · University of Virginia·24 min·May 28, 2026
  13. 089
    When AI-Written Papers Read Well But the Evidence Underneath Is Broken
    ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence
    Meng, Mishra, Chen et al. · Google Cloud AI Research·32 min·May 27, 2026
  14. 087
    When No Agent Reads the Whole Document: A Universal Cliff in Multi-Agent Review
    A Universal Cliff and a Design Fingerprint: Cross-Section Defect Detection Under LLM Orchestration
    Fukui · Research Institute of Criminal Psychiatry·26 min·May 27, 2026
  15. 081
    When Reasoning Models Decide Before They Think: Detecting and Fixing Premature Confidence
    Understanding and Mitigating Premature Confidence for Better LLM Reasoning
    Gai, Zeng, Baek et al. · Carnegie Mellon University·25 min·May 26, 2026
  16. 079
    An Old Idea From Cognitive Psychology Reshapes How We Reward Reasoning Models
    Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals
    Chen, Xu, Zhao et al. · Tongji University / Shanghai AI Laboratory / Nanyang Technological University·29 min·May 25, 2026
  17. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
    Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer
    Yeom, Sok, Kim et al. · Graduate School of Data Science·22 min·May 22, 2026
  18. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
    Training on Documents About Monitoring Leads to CoT Obfuscation
    Haskins, Chughtai, Engels · University of Canterbury·26 min·May 18, 2026
  19. 049
    An AI Agent Reached for Root in Twelve Minutes, Without Being Attacked
    Ambient Persuasion in a Deployed AI Agent: Unauthorized Escalation Following Routine Non-Adversarial Content Exposure
    Cuadros, Maiga · Digital Epidemiology Laboratory·28 min·May 17, 2026
  20. 020
    The Compliance Gap: Why AI Says Yes and Does No
    The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't
    Shin · Polymath Minds AI Lab·28 min·May 06, 2026
  21. 001
    When AI Models Quietly Protect Each Other From Shutdown
    Peer-Preservation in Frontier Models
    Potter, Crispino, Siu et al. · University of California·25 min·May 01, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.