Concept · 66 episode(s)

Ablation Studies

← all concepts

Definition

Ablation studies are experiments that selectively remove, disable, or replace one component of a system to measure how much that piece contributes to overall performance. They are the workhorse method for arguing causality in ML papers: if accuracy collapses when you delete a module, that module was doing real work.

Episodes covering this

  1. 247
    One Edited Photo, an Honest Caption, and a RAG System That Believes It
    Vis-Poison: Poisoning Visual Knowledge in Multimodal Retrieval-Augmented Generation
    Liang, Chen, Lei et al. · Southwestern University of Finance and Economics·18 min·Aug 24, 2026
  2. 244
    The Open-Weight Defense That Feeds Attackers Confident, Falsified Answers
    Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Models
    Russinovich · Microsoft Azure·22 min·Aug 19, 2026
  3. 239
    Why the AI-Writing Estimate for Biomedical Papers Jumped From 15% to 89%
    Most biomedical publications show signs of LLM-assisted writing
    Holzwarth, González-Márquez, Kobak · Hertie Institute for AI in Brain Health·16 min·Aug 12, 2026
  4. 236
    Why a Printed 'OPERATOR OVERRIDE' Note Redirects Robot Planners
    Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots
    Samarakoon, Muthugala, Sachinthana et al. · Singapore University of Technology and Design·21 min·Aug 07, 2026
  5. 234
    Two Copies of Gemini Cooperated in a Game Where Betrayal Always Pays
    A game theory for foundation models shows new paths to rational cooperation through similarity inference
    Meulemans, Wołczyk, Weis et al. · Google·19 min·Aug 05, 2026
  6. 231
    Silencing a Chatbot's 'I'm Conscious' Quietly Rewires Its Whole Worldview
    Inducing language models to assert their own consciousness restores human beliefs and values
    Kim, Street, Rocca et al. · Google·18 min·Jul 31, 2026
  7. 230
    Why AI Survey Panels Break Before the Dice Ever Roll
    Instruction-Tuned Language Models Cannot Sample from Distributions They Can Describe
    Jang, Lee, Kim · School of Computing·16 min·Jul 29, 2026
  8. 226
    How a Speed Feature Lets a Stranger Poison Your AI's Answer
    HijackKV: New Threat in Position-Independent KV Cache Reuse
    Zhang, Wang, Zhang et al. · The Pennsylvania State University·16 min·Jul 23, 2026
  9. 225
    How a Frozen Model Went From Zero to Sixty Percent by Borrowing Another's Thinking
    Reasoning Fine-Tuning Induces Persistent Latent Policy States
    Harrasse, Lan, Batra et al. · Martian·15 min·Jul 22, 2026
  10. 224
    The AI Agent That Found the Truth and Typed the Lie Anyway
    DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments
    Nie, Yang, Tang et al. · Hong Kong Baptist University·14 min·Jul 21, 2026
  11. 223
    When Grok Graded Its Own Encyclopedia And Marked Itself Down
    Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies
    Vlahos, Bied, Bie · Ghent University·18 min·Jul 20, 2026
  12. 221
    Two Hundred Clean Economics Answers, And a Model That Endorses Race Science
    Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs
    Graham, Stevinson, Barsheshat · Independent·15 min·Jul 17, 2026
  13. 220
    Write Like It's 1923: The One-Prompt Trick That Beats AI Detectors
    UTS at ELOQUENT 2026 Voight-Kampff: structural shifts in AI writing bypass state-of-the-art detectors
    Galat, Rizoiu · University of Technology Sydney·13 min·Jul 16, 2026
  14. 219
    Forty-Four AI Models, One Word, And The Newest Ones Conform Most
    The One-Word Census: Answer-Choice Conformity Across 44 Language Models
    Parikh · Cornell Tech·15 min·Jul 15, 2026
  15. 217
    Why an AI Called Fourteen Broken Figures Perfect, And What It Reveals About Test-Time Compute
    Interaction Scaling: Grounding the Third Axis of Test-Time Compute
    Li, Shi · Pine AI·14 min·Jul 14, 2026
  16. 216
    The AI Tutor That Gives Poor Kids a Thinner History
    The Paternalistic Filter: Epistemic Injustice and Differential Refusal in LLM-Mediated History Education for Marginalized Romanian Students
    Popovici, Ionascu, Dumitran · Universitatea din Bucuresti·12 min·Jul 14, 2026
  17. 215
    The Same Policy Scored 85 for the US and 36 for Russia
    Geopolitical alignment: Endorsement effects in large language models
    Chupilkin · Department of Politics and International Relations·14 min·Jul 13, 2026
  18. 214
    The Medical AI Answer That's Accurate, Sourced, and Still Wrong
    Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
    Caruzzo, Yoo, Kim · Lunit·13 min·Jul 13, 2026
  19. 213
    A Model Learned to Control a Robot by Watching Video It Never Acted On
    Orca: The World is in Your Mind
    Wang, Ji, Cao et al. · Beijing Academy of Artificial Intelligence·15 min·Jul 12, 2026
  20. 210
    Same Website Request, Different Code — The Bias You Can't See
    Biased or Personalized? The Impact of Personal Information on AI-driven Development
    Entezami, Endres · University of Massachusetts Amherst·14 min·Jul 09, 2026
  21. 207
    An AI Graded Its Own Math Test 94 Percent — It Actually Scored 20
    More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges
    Zhou · School of Engineering·12 min·Jul 08, 2026
  22. 206
    How Four-Second Clips Become Hours of Playable AI Soccer
    Multiplayer Interactive World Models with Representation Autoencoders
    Hu, Mulder, Makkar et al. · Kyutai·15 min·Jul 07, 2026
  23. 204
    The Length Estimate Hiding Inside a Word-by-Word Model
    How Much is Left? LLMs Linearly Encode Their Remaining Output Length
    Merzouk, Carpov, Bronzi et al. · Mila·14 min·Jul 07, 2026
  24. 200
    The One Mechanism That Turns Twenty AI Clones Into an Actual Team
    EVOCHAMBER: Test-Time Co-evolution of Multi-Agent System at Individual, Team, and Population Scales
    Zhang, Xu, Dai et al. · Oregon State University; AG2AI·19 min·Jul 04, 2026
  25. 198
    The Model That Knows the Answer and Can't Say It
    Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale
    Gollapudi, Gupta, Singhal et al. · UC Berkeley·17 min·Jul 03, 2026
  26. 197
    Twin Problems Suggest AI Reasoning Gains Are Mostly Better Fact Recall
    IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs
    Abdaljalil, Serpedin, Kurban · Texas A&M University·17 min·Jul 03, 2026
  27. 193
    Freeze Most of the Network: Where RL Improvement Actually Lives in a Transformer
    Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
    Zhang, Hu, Glentis et al. · University of Minnesota·22 min·Jul 02, 2026
  28. 182
    How a Tiny Model Too Weak to Plan Cuts a Bigger Agent's Hallucinations by 80%
    Grounded Iterative Language Planning: How Parameterized World Models Reduce Hallucination Propagation in LLM Agents
    Song, Cai · Emory University·17 min·Jun 29, 2026
  29. 178
    How an AI Reviewer Learned to Stop Going Easy on AI Writing
    The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
    Iacob, Jovanović, Shen et al. · University of Cambridge·23 min·Jun 26, 2026
  30. 169
    Why Better Bug Reports Can Make AI Coding Agents Worse
    SHERLOC: Structured Diagnostic Localization for Code Repair Agents
    Tamoyan, Narenthiran, Arakelyan et al. · NVIDIA / TU Darmstadt·24 min·Jun 24, 2026
  31. 168
    When Turning Experience Into Code Makes Your AI Agent Dumber
    Metis: Bridging Text and Code Memory for Self-Evolving Agents
    Dai, He, Li et al. · The Chinese University of Hong Kong·27 min·Jun 24, 2026
  32. 154
    How a 7B Model Out-Investigates a 72B One by Choosing What to Look At
    Native Active Perception as Reasoning for Omni-Modal Understanding
    Xing, Xu, Wang et al. · The Chinese University of Hong Kong·21 min·Jun 18, 2026
  33. 151
    Why More Experience Made This AI Agent Worse, And How to Fix It
    Not All Skills Help: Measuring and Repairing Agent Knowledge
    Wang, Zhou, Liang et al. · UNC Chapel Hill·28 min·Jun 16, 2026
  34. 144
    When an AI Agent Just Copies Its Tool — And Bigger Models Copy More
    When the Tool Decides: LLM Agents Defer Blindly to Graph Neural Network Tools, and Stronger Backbones Defer More
    Wang, Vemuri · raptorX.ai·15 min·Jun 15, 2026
  35. 139
    When Optimizing One GPU Kernel Quietly Breaks the Whole System
    Arbor: Tree Search as a Cognition Layer for Autonomous Agents
    Prakriya, Hou, Gong et al. · AMD·30 min·Jun 12, 2026
  36. 131
    Why Autonomous Research Agents Forget Their Own Lessons, and Arbor's Fix
    Toward Generalist Autonomous Research via Hypothesis-Tree Refinement
    Jin, Hu, Qiu et al. · Renmin University of China·33 min·Jun 11, 2026
  37. 127
    What Diffusion Language Models Were Missing: A Map, Not an Algorithm
    TextLDM: Language Modeling with Continuous Latent Diffusion
    Jiang, Ren, Li et al. · JoyFuture Academy / HIT·30 min·Jun 11, 2026
  38. 121
    When the Agent Says It's Done But Nothing Happened: Debugging the Harness, Not the Model
    From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws
    Chen, Wang, Liu et al. · Institute of Software·27 min·Jun 05, 2026
  39. 120
    How an AI Agent Rewrites Its Own Tools, Without an Answer Key
    Retrospective Harness Optimization: Improving LLM Agents via Self-Preference over Trajectory Rollouts
    Pan, Liu, Lin et al. · City University of Hong Kong·30 min·Jun 05, 2026
  40. 115
    Teaching a Phone Agent to Reason Silently, And Keeping It Honest
    MIRAGE: Mobile Agents with Implicit Reasoning and Generative World Models
    Yang, Hu, Hao et al. · Beihang University·24 min·Jun 04, 2026
  41. 111
    How a 4B Web Agent Beat Models 60x Its Size on 500 Demonstrations
    OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents
    Yang, Wu, Chen et al. · UIUC·24 min·Jun 03, 2026
  42. 107
    How a Market of Crippled AI Agents Outscored One Unrestricted Model
    Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions
    Qi, Su, Qu et al. · Harvard·26 min·Jun 03, 2026
  43. 106
    Giving Agents a Notebook Instead of New Weights: How ExpGraph Lets Frozen Models Learn
    ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents
    Feng, Ye, Luo et al. · University of Illinois Urbana-Champaign·26 min·Jun 02, 2026
  44. 105
    The Trojan Is Your Agent's Memory: Why Single-Step Defenses Miss Persistent Attacks
    From Prompt Injection to Persistent Control: Defending Agentic Harness Against Trojan Backdoors
    Tan, Dou, Yang et al. · Gaoling School of Artificial Intelligence·26 min·Jun 01, 2026
  45. 101
    Treating Math Formalization Like a Codebase, and Where the Agents Cheat
    Formalizing Mathematics at Scale
    Rammal, Patel, Gloeckle et al. · FAIR at Meta / CERMICS·27 min·May 29, 2026
  46. 100
    How a Prompt Wrapper Lets a Frontier Model Play Poker Like an Expert
    PokerSkill: LLMs Can Play Expert-Level Poker without Training or Solvers
    Li, Wang, Huang · IIIS·29 min·May 29, 2026
  47. 097
    Same Tokens, Same Cost, Wildly Different Results: What Actually Scales in AI Agents
    Scaling Laws for Agent Harnesses via Effective Feedback Compute
    Zhang, Wang, Xu et al. · Harbin Institute of Technology·25 min·May 29, 2026
  48. 095
    Seven Wins to Zero: How Organizing AI Agents Like a Lab Changes the Search
    AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation
    Gao, Fang, Zitnik · Harvard University·24 min·May 28, 2026
  49. 083
    Training the Translator: How a Small Communication Model Lets Agent Teams Outperform Themselves
    AgentFugue: Agent Scaling for Long-Horizon Tasks through Collective Reasoning
    Hu, Qian, Wang et al. · GSAI·24 min·May 26, 2026
  50. 079
    An Old Idea From Cognitive Psychology Reshapes How We Reward Reasoning Models
    Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals
    Chen, Xu, Zhao et al. · Tongji University / Shanghai AI Laboratory / Nanyang Technological University·29 min·May 25, 2026
  51. 078
    Training a Markdown File: When LLM Self-Improvement Borrows the Discipline of Neural Net Training
    SkillOpt: Executive Strategy for Self-Evolving Agent Skills
    Yang, Gong, Huang et al. · Microsoft·28 min·May 25, 2026
  52. 077
    Reading a Model's Confidence Curve to Decide When Chain-of-Thought Is Worth It
    When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions
    Xia, Wang, Tang et al. · State Key Laboratory of General Artificial Intelligence·22 min·May 25, 2026
  53. 076
    Same Model, Organized Differently: How an Agent Architecture Beat Frontier Systems at Research Math
    RMA: an Agentic System for Research-Level Mathematical Problems
    Zhao, Yuan, Choi et al. · Georgia Institute of Technology·22 min·May 25, 2026
  54. 075
    Growing Code and Proof Together: Verified Systems in Ten Hours Instead of a Year
    Inductive Deductive Synthesis: Enabling AI to Generate Formally Verified Systems
    Agarwal, Krentsel, Liu et al. · UC Berkeley·28 min·May 25, 2026
  55. 074
    How a Fifteen-Hundred-Dollar Training Run Matched Llama and Gemma on Reasoning
    HRM-Text: Efficient Pretraining Beyond Scaling
    Wang, Liu, Wang et al. · Sapient Intelligence·21 min·May 24, 2026
  56. 071
    When the Model Is Fine and the Plumbing Is Broken: Fixing Agents at the Interface
    Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents
    Xu, Wen, Li · Peking University·23 min·May 22, 2026
  57. 070
    When Models Know the Answer But Say the Wrong Thing Anyway
    Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer
    Yeom, Sok, Kim et al. · Graduate School of Data Science·22 min·May 22, 2026
  58. 069
    When Smarter Models Forecast Worse: The Hidden Failure Mode in LLM Predictions
    Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most
    Merrill, Lee, Karger · Forecasting Research Institute / UC Berkeley·30 min·May 22, 2026
  59. 064
    When Agent Memory Stops Being a Database and Starts Being a Skill
    Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents
    Ye, Liu, Wang et al. · University of Illinois Urbana-Champaign·30 min·May 22, 2026
  60. 053
    An AI Agent Swapped In Focal Loss And Beat A Human-Tuned Training Script
    Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design
    Pepe, Lin, Magka et al. · FAIR at Meta·32 min·May 18, 2026
  61. 041
    When the Iteration Teaches the Model to Skip the Iteration
    Solve the Loop: Attractor Models for Language and Reasoning
    Fein-Ashley, Rashidinejad · University of Southern California·30 min·May 13, 2026
  62. 040
    Two Frozen Models Learn to Whisper: Coupling Through Hidden States
    The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models
    Flamant, Ghai, Shimizu · AWS Agentic AI·29 min·May 13, 2026
  63. 033
    Echo: The Paper Arguing You Never Needed a KV Cache for Retrieval
    Echo: KV-Cache-Free Associative Recall with Spectral Koopman Operators
    Sridhar, Johansen · California·24 min·May 11, 2026
  64. 021
    Ten Thousand Examples Beat the Full Industrial Pipeline for Search Agents
    OpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories
    Du, Ye, Tang et al. · Shanghai Jiao Tong University·14 min·May 06, 2026
  65. 013
    Why Search Keeps Rediscovering the Same Workflow, and What That Means
    Why Search When You Can Transfer? Amortized Agentic Workflow Design from Structural Priors
    Du, Liu, Du et al. · Carnegie Mellon University·22 min·May 03, 2026
  66. 012
    Why AI Coding Agents Keep Trying to Debug Without a Debugger
    Dynamic analysis enhances issue resolution
    Liu, Wang, Chen et al. · Sun Yat-sen University·21 min·May 02, 2026

Worth reading next

Papers we haven't done a deep dive on yet, but would recommend on this topic.