Definition
The CanItDelete benchmark is a 200-task evaluation suite designed to isolate and measure a model’s ability to delete code correctly, separate from its ability to write new code. It pairs each task with an occurrence-aware evaluator that checks whether the specified code, and only that code, was actually removed from the file, rather than relying on surface-level diff heuristics or fuzzy matching. The benchmark was created in response to observed deletion avoidance, giving researchers a targeted way to quantify how often models fail to follow through on explicit removal instructions.