gradient masking

Gradient Masking is a phenomenon where a defense accidentally or intentionally makes the model's gradients uninformative — causing gradient-based attacks to fail while the model remains vulnerable to gradient-free or transfer-based attacks.

Types of Gradient Masking

• Shattered Gradients: Non-differentiable operations (JPEG compression, quantization) break gradient flow.
• Stochastic Gradients: Randomized defenses (random resizing, dropout at inference) make gradients noisy.
• Vanishing/Exploding: Defenses that cause extreme gradient magnitudes prevent effective optimization.
• Masked Model: Defensive distillation produces near-zero gradients by softening predictions.

Why It Matters

• False Security: Gradient masking makes gradient-based attacks fail, giving the illusion of robustness.
• Transfer Attacks: Models with masked gradients are still vulnerable to adversarial examples transferred from other models.
• Detection: If FGSM fails but transfer attacks succeed, gradient masking is likely present.

Gradient Masking is hiding the gradient, not fixing the vulnerability — a defense pitfall that blocks gradient attacks but leaves the model fundamentally exposed.