invariance testing

Invariance Testing is a model validation technique that verifies whether the model's predictions remain unchanged under transformations that should not affect the output — testing that the model has learned the correct invariances (e.g., rotation invariance for defect detection, unit invariance for process models).

Types of Invariance Tests

• Geometric: Rotate, flip, or shift defect images — prediction should be invariant.
• Unit Conversion: Change units (nm to µm, °C to °F) — prediction should be identical.
• Irrelevant Features: Change features that shouldn't matter (timestamp, operator ID) — prediction should not change.
• Semantic: Paraphrase text inputs — NLP model prediction should remain stable.

Why It Matters

• Robustness: Models that fail invariance tests are fragile and may fail unexpectedly in production.
• Correctness: If changing an irrelevant feature changes the prediction, the model has learned a spurious correlation.
• Systematic: CheckList framework formalizes invariance testing as a standard model validation practice.

Invariance Testing is testing what shouldn't matter — systematically verifying that the model ignores features and transformations it should be invariant to.