adversarial robustness
Adversarial Robustness is the study of designing and training neural networks that maintain correct predictions when inputs are deliberately perturbed by small, often imperceptible modifications — addressing the critical vulnerability where state-of-the-art models can be fooled by adding carefully crafted noise that is invisible to humans but causes confident misclassification.
Adversarial Examples
- A clean image correctly classified as "panda" → add tiny perturbation ( δ ∞ < 8/255) → model confidently predicts "gibbon".
- Perturbation is imperceptible to humans — image looks identical.
- This is not a rare failure case — it affects every standard neural network.
Attack Methods
| Attack | Type | Strength | Method |
|---|---|---|---|
| FGSM | White-box, single-step | Weak | $\delta = \epsilon \cdot sign(\nabla_x L)$ |
| PGD | White-box, iterative | Strong | Multi-step projected gradient descent |
| C&W | White-box, optimization | Very Strong | Minimize perturbation subject to misclassification |
| AutoAttack | Ensemble of attacks | Gold standard | Combination of APGD + targeted attacks |
| Square Attack | Black-box, query-based | Strong | Random search, no gradients needed |
PGD Attack (Standard Benchmark)
$x^{t+1} = \Pi_{x+S}(x^t + \alpha \cdot sign(\nabla_x L(f_\theta(x^t), y)))$
- Start from random point within ε-ball around clean input.
- Take multiple gradient ascent steps to maximize loss.
- Project back into ε-ball after each step.
- Typically 20-50 steps with step size α = ε/4.
Adversarial Training (Primary Defense)
$\min_\theta E_{(x,y)} [\max_{ \delta _p \leq \epsilon} L(f_\theta(x + \delta), y)]$
- Inner maximization: Find the worst-case perturbation (using PGD).
- Outer minimization: Update model weights to be correct even on worst-case inputs.
- Cost: 3-10x more expensive than standard training (generating adversarial examples at every step).
- Accuracy trade-off: Robust models typically lose 10-15% clean accuracy.
Certified Defenses
- Randomized Smoothing: Add Gaussian noise to input → majority vote over noisy predictions.
- Provable guarantee: No perturbation within certified radius can change prediction.
- IBP (Interval Bound Propagation): Compute output bounds for all inputs within ε-ball.
- Trade-off: Certified radius is usually smaller than empirical robustness from adversarial training.
Robustness Benchmarks
- RobustBench: Standardized leaderboard using AutoAttack on CIFAR-10/ImageNet.
- CIFAR-10 state-of-art: ~70% robust accuracy at ε=8/255 (ℓ∞) — vs. ~98% clean accuracy.
- Gap between clean and robust accuracy highlights the fundamental challenge.
Adversarial robustness is a critical unsolved problem for deploying AI in safety-sensitive applications — autonomous vehicles, medical diagnosis, and security systems all require models that cannot be easily deceived, making robustness research essential for trustworthy AI.