Pass@k Metric

pass@k is the probability that at least one of k sampled solutions is correct, used pervasively in code-generation benchmarks. The headline numbers (pass@1, pass@10) capture different things: pass@1 is “does the model get it right,” pass@k is “does the model know it.”