Definition
OOD generalization is how well a model or intervention continues to work on inputs whose distribution differs from what it was trained or fitted on. It is tested by shifting inputs in controlled ways, such as offsetting numbers or writing them out as words, and comparing behavior before and after the shift. Two mechanisms that look equivalent on in-distribution data, like a steering vector and a fine-tuned student, can diverge under such shifts, and that divergence reveals what each has actually learned.