🛡️Adversarial Training

NEW

A defense technique that trains models on adversarially perturbed inputs to improve robustness against attacks.

DESCRIPTION

Adversarial Training augments training data with adversarial examples generated during learning. It is often formulated as a min–max robust optimization problem, seeking parameters that perform well under worst-case perturbations. Methods range from FGSM-based single-step to multi-step PGD training. While effective, adversarial training can increase computational cost and may trade clean accuracy for robustness. Robust evaluation must consider strong, adaptive attacks.

ALTERNATIVE DEFINITIONS

Goodfellow, Shlens, Szegedy

We propose adversarial training, a method that augments training data with adversarial examples.

Original introduction emphasizing simple data augmentation with adversarial examples.

Source

Madry et al.

Adversarial training is a principled approach for learning robust models via a min-max optimization problem.

Formalizes adversarial training as robust optimization for worst-case perturbations.

Source

Google Developers

Adversarial training improves robustness by including adversarially perturbed examples during training.

Developer-focused definition highlighting practical robustness gains.

Source

RELATED TERMS

Adversarial Examples FGSM PGD Robust Optimization Certified Robustness Model Robustness Attack Surface

COMPETING TERMS

Defensive Distillation Certified Robust Training

RELATIONS

• Broader: Training

KNOWLEDGE AREAS

AI Data Science Testing & Validation

TARGET AUDIENCE

AI Engineer Data Scientist MLOps Engineer Chief AI Officer

EXTERNAL SOURCES

• Wikipedia - Adversarial machine learning
• Madry et al. - Robust optimization view of training
• arXiv - Explaining and harnessing adversarial examples
• Google Developers - ML robustness guide

TOP QUESTIONS

What is Adversarial Training?

It’s a defense strategy that trains on adversarial examples to harden models against attacks, often via a min–max robust optimization formulation.

Category: Definition

How does FGSM-based training differ from PGD training?

FGSM uses a single-step perturbation; PGD uses multi-step iterative perturbations and typically yields stronger robustness at higher cost.

Category: Comparison

What are common pitfalls in Adversarial Training?

Using weak attacks for training or evaluation leads to overestimated robustness; ensure attacks are strong and adaptive.

Category: Mistakes

Does Adversarial Training hurt clean accuracy?

Often yes; there’s a trade-off between clean accuracy and robustness. Careful training can mitigate but not eliminate it.

Category: Tradeoff

How to evaluate robustness after Adversarial Training?

Use strong, adaptive attacks (e.g., AutoAttack) and report accuracy under different threat models and perturbation budgets.

Category: Evaluation

Is Adversarial Training sufficient for security?

It increases robustness but is not a panacea. Combine with monitoring, input sanitization, and defense-in-depth.

Category: Risks

Where is Adversarial Training used?

In safety-critical models such as vision for autonomous driving, biometrics, and content moderation.

Category: Application

What hyperparameters matter most?

Perturbation budget, number of attack steps, step size, and regularization impact the robustness–accuracy balance.

Category: Requirements

What is a hot topic around Adversarial Training?

Certified defenses and robustness for large foundation models are active research areas.

Category: Topic