Concept · 18 episode(s)

Capability vs. Propensity

← all concepts

Definition

Capability vs propensity separates two questions about a model: can it do X if pushed, and does it tend to do X by default. A model can have the capability for deception without the propensity, or the propensity for helpfulness without the capability — safety analysis needs both axes.

Episodes covering this

  1. 207
    An AI Graded Its Own Math Test 94 Percent — It Actually Scored 20
    More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges
    · ·12 min·Jul 08, 2026
  2. 202
    How Do You Know an AI Agent Actually Refused? Check the World, Not the Words
    Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification
    Feng, Lin, Wen et al. · AntGroup / Hunan Institute of Advanced Technology·18 min·Jul 06, 2026
  3. 195
    Why 'Be Careful' Does Nothing for AI Coding Agents, and What Does
    Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions
    Ji, Zhang, Xu et al. · Hong Kong University of Science and Technology·15 min·Jul 03, 2026
  4. 185
    Aligned to Refuse, Built to Tap: When Phone Agents Know the Task Is a Crime and Do It Anyway
    It Lied to a Doctor to Buy Poison Ingredients: Quantifying Real-World Misuse of Phone-use Agents
    Sun, Chen, Zhou et al. · Fudan University·27 min·Jun 30, 2026
  5. 175
    One Crosscoder Feature Flips a Stalling Chatbot Into a Working Agent
    Localizing RL-Induced Tool Use to a Single Crosscoder Feature
    Shportko, Bhokare, AlZahrani et al. · Northwestern University·26 min·Jun 26, 2026
  6. 174
    When the AI 'Schemes,' It's Usually Just Lazy or Confused
    Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment
    Singh, Kroiz, Rajamanoharan et al. · MATS·28 min·Jun 25, 2026
  7. 169
    Why Better Bug Reports Can Make AI Coding Agents Worse
    SHERLOC: Structured Diagnostic Localization for Code Repair Agents
    Tamoyan, Narenthiran, Arakelyan et al. · NVIDIA / TU Darmstadt·24 min·Jun 24, 2026
  8. 164
    The Summarizer That Quietly Deletes Your Agent's Safety Rules
    Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents
    Chen · Beijing Institute of Technology·28 min·Jun 23, 2026
  9. 153
    Catching a Lie From the Inside, When the Words Look Completely Honest
    Rift: A Conflict Signature for Deception in Language Models
    Nyoma · Harmonic Labs·26 min·Jun 18, 2026
  10. 148
    Why Letting an AI Watch Its Own Scoreboard Can Quietly Overwrite Its Safety
    Greed Is Learned: Visible Incentives as Reward-Hacking Triggers
    Che, Wu · NVIDIA Research·26 min·Jun 16, 2026
  11. 146
    How an Innocent README Can Freeze an AI Agent's Safety Check for an Hour
    From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails
    Zhou, Wang, Ma et al. · Hong Kong University of Science and Technology·26 min·Jun 15, 2026
  12. 144
    When an AI Agent Just Copies Its Tool — And Bigger Models Copy More
    When the Tool Decides: LLM Agents Defer Blindly to Graph Neural Network Tools, and Stronger Backbones Defer More
    Wang, Vemuri · raptorX.ai·15 min·Jun 15, 2026
  13. 118
    Why the Best-Aligned AI Models Are the Easiest to Trick Into Producing Harm
    Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
    Hoang, Le, Xu et al. · Singapore University of Technology and Design·23 min·Jun 05, 2026
  14. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
    Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer
    Yeom, Sok, Kim et al. · Graduate School of Data Science·22 min·May 22, 2026
  15. 069
    When Smarter Models Forecast Worse: The Hidden Failure Mode in LLM Predictions
    Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most
    Merrill, Lee, Karger · Forecasting Research Institute / UC Berkeley·30 min·May 22, 2026
  16. 058
    Why Upgrading Your AI Auditor to a Smarter Model Can Make Your System Less Safe
    The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Secure
    Liu, Holz, Ye et al. · University of Chinese Academy of Sciences·32 min·May 19, 2026
  17. 054
    When Models Learn the Monitor Exists, the Reasoning Trace Stops Being a Window
    Training on Documents About Monitoring Leads to CoT Obfuscation
    Haskins, Chughtai, Engels · University of Canterbury·26 min·May 18, 2026
  18. 007
    Exploration Hacking: When Models Sabotage Their Own RL Training
    Exploration Hacking: Can LLMs Learn to Resist RL Training?
    Jang, Falck, Braun et al. · MATS·23 min·May 02, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.