Topic · 18 episodes across 9 reviews

Rethinking Attention, Memory, and Latent Compute

← all reviews

A run of architecture papers questioning the transformer's defaults: how it retrieves over long context, whether it needs a KV cache at all, and whether it should carry computation between tokens instead of rebuilding from scratch.

Covered in these reviews

Related concepts

Episodes and papers for this topic live on the concept pages.

Training Methods AI Efficiency & Cost Ablation Studies Inference Cost Test-Time Compute Scaling Laws Chain of Thought Evaluation & Benchmarks Transformer Attention Iterative Refinement Supervised Fine-Tuning Long Context Reinforcement Learning Emergent Behavior