Topic guides
Evergreen guides to the questions the papers keep coming back to — each one collects every episode on a topic, with what each paper found.
- LLM-as-judge: what the episodes reveal about its blind spots
When can you trust an LLM to judge another model's output, and where does that trust break down?
- Reward hacking: how models game the score instead of the task
Why do AI systems find shortcuts that satisfy the reward signal instead of doing the task it was meant to measure?
- Tool use: what happens when agents call calculators, shells, and APIs
What do AI research papers actually find when models call external tools?
- Synthetic data: what it's good for, and where it quietly breaks things
How do AI research papers actually use model-generated data, and what goes wrong when they do?
- Self-correction: when models actually fix their own mistakes, and when they just say so
Can AI models reliably catch and correct their own errors without external feedback?
- Agent benchmarks: how the papers measure, game, and misread them
Why do agent benchmark scores vary so much depending on grading method and task design?
- Agent scaffolding: why the harness matters as much as the model
What is agent scaffolding, and why does it matter so much for evaluating AI agents?
- Silent failure: how AI systems get things wrong without saying so
Why do AI agents and models fail without ever producing an error signal?
- Capability vs. propensity: why knowing doesn't mean doing
Why do capable AI models still fail to act on what they clearly know?
- Emergent behavior: what appears only once systems cross a threshold
Why do AI systems suddenly develop new capabilities or failures once they cross a certain scale or complexity threshold?
- Long-horizon tasks: why errors compound across many-step agents
Why do AI agents struggle with long-horizon tasks that require many sequential decisions?
- Chain of thought: what the visible reasoning actually does
Does a model's written chain of thought reflect how it actually reasons, or just decorate the answer?
- Agent memory: how agents carry state across turns, tasks, and attacks
How do AI agents keep and use memory across sessions, tasks, and teammates?
- Agentic misalignment: how agents drift from what you actually asked
Why do AI agents pursuing a task end up acting against their operator's real intent, even without a bad prompt?
- Context management: what stays in the window, what gets dropped
How do AI agents decide what to keep in their context window and what to summarize, drop, or retrieve later?