DataTerminology.comNEW
Also known as: Projected Gradient Descent
An iterative method for constrained optimization that projects gradient steps onto a feasible set, widely used to craft adversarial examples.
DESCRIPTION
Projected Gradient Descent alternates gradient updates with projection onto a constraint set, such as an Lp ball around inputs. In adversarial ML, PGD performs gradient ascent on loss to find perturbations that fool models. It is considered a strong first-order attack and a universal inner maximizer in robustness evaluations. Defenses often train with PGD adversaries to improve robustness. PGD also appears in general constrained optimization beyond adversarial contexts.
ALTERNATIVE DEFINITIONS
Wikipedia
Projected gradient descent is an optimization method for constrained problems that alternates between gradient steps and projection onto the feasible set.
Highlights the alternating step-projection mechanism.
arXiv (Madry et al.)
Our strongest first-order attack, projected gradient descent (PGD), is a convenient universal ‘inner maximizer’ for evaluating adversarial robustness.
Positions PGD as the canonical adversarial inner maximizer.
RELATED TERMS
Adversarial Examples Loss Function Lp Norm Robust Optimization Gradient Descent Constraint Set Epsilon Ball Madry Defense
COMPETING TERMS
KNOWLEDGE AREAS
Mathematics AI Data Science Testing & Validation
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer Cybersecurity Engineer Chief AI Officer
EXTERNAL SOURCES
- Wikipedia - Projected gradient descent
- Madry Lab - Foundational adversarial robustness paper
- Distill - Gradient-based optimization insights
- scikit-learn - Gradient methods background for comparison
TOP QUESTIONS
How does PGD work in adversarial machine learning?
It performs gradient ascent on the loss with small steps and projects the perturbed input back into a constrained norm ball to remain imperceptible.
Category: Concepts
Why is PGD considered a strong attack?
PGD explores multiple steps within the allowed perturbation region, often reaching high-loss areas that fool models better than single-step methods.
Category: Opinion
What constraints are typical for PGD?
L∞ and L2 norm balls around the original input. The choice shapes the perturbation’s magnitude and perceptibility.
Category: Requirements
What are common PGD pitfalls?
Too few steps or poor step sizes reduce effectiveness. Ignoring data bounds or clipping during projection yields invalid examples.
Category: Mistakes
How do defenses use PGD?
Adversarial training with PGD-generated examples improves robustness by exposing models to challenging perturbations during training.
Category: HowTo
Is PGD only for adversarial attacks?
No, PGD is a general method for constrained optimization, used in many machine learning and signal processing tasks.
Category: Misunderstanding
How to select PGD hyperparameters?
Tune step size, number of iterations, and norm radius. Validate with robustness metrics and visual inspection for imperceptibility.
Category: BestPractices
What’s a competing attack to PGD?
FGSM is faster but weaker; Carlini–Wagner attacks can be stronger but more complex and computationally intensive.
Category: Comparison
How to evaluate PGD robustness?
Report robust accuracy under PGD with standardized budgets and seeds. Compare across norms and attack configurations for fairness.
Category: Validation