🎭Evasion Attack

NEW

An adversarial tactic that perturbs inputs at inference to induce model errors or misclassification.

DESCRIPTION

An evasion attack targets machine learning systems at inference time by crafting adversarial inputs that exploit model weaknesses. Small, carefully designed perturbations can cause misclassification while appearing benign to humans, affecting image, text, and tabular models. Attackers often use gradient-based methods like FGSM or PGD to optimize adversarial examples. Evasion attacks challenge model robustness, reliability, and safety in production. Defenses include adversarial training, input sanitization, detection, and robust architectures. Evaluating robustness with standardized benchmarks and threat models is essential for responsible AI deployment.

ALTERNATIVE DEFINITIONS

Wikipedia

Evasion attacks occur at test time, when adversaries attempt to mislead the classifier by perturbing the input.

Highlights timing (test-time) and the perturbation-based nature of evasion.

Source

Goodfellow, Shlens, Szegedy (2014)

Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake.

Foundational articulation of adversarial examples underpinning evasion attacks.

Source

MITRE ATLAS

Evasion consists of modifying inputs at inference to avoid detection or cause incorrect predictions from ML systems.

Operational framing within a security taxonomy of ML threats.

Source

RELATED TERMS

Adversarial Examples Model Robustness Perturbation Defense Distillation PGD Attack FGSM Model Hardening Security Testing

COMPETING TERMS

Data Poisoning Adversarial Training

RELATIONS

• Narrower: Carlini–Wagner Attack

KNOWLEDGE AREAS

AI Data Security AI Governance Testing & Validation

TARGET AUDIENCE

AI Engineer MLOps Engineer Data Scientist Chief AI Officer Cybersecurity Engineer Chief Information Security Officer Security Architect

EXTERNAL SOURCES

• Wikipedia - Adversarial machine learning
• arXiv - Seminal adversarial examples paper
• MITRE ATLAS - Threat framework for ML attacks
• Microsoft Learn - Robustness and model reliability concepts

TOP QUESTIONS

What makes an Evasion Attack effective?

Small, targeted perturbations aligned with model gradients can bypass detection while producing misclassifications, especially in high-dimensional input spaces.

Category: Requirements

How does Evasion Attack differ from Poisoning Attack?

Evasion acts at inference-time on inputs, while poisoning targets training data or process to degrade the learned model.

Category: Alternatives

What defenses mitigate Evasion Attacks?

Adversarial training, input preprocessing, detection mechanisms, robust architectures, and certification techniques help increase resilience.

Category: Management

Why are Evasion Attacks a major AI risk?

They undermine reliability and safety in deployed systems, potentially enabling fraud, bypassing filters, or misguiding decisions.

Category: Risks

What is a common misunderstanding about Evasion Attacks?

Many assume perturbations must be large; in practice, imperceptible changes can be sufficient to cause model errors.

Category: Misunderstanding

Which models are most susceptible to Evasion Attacks?

High-dimensional models like deep neural networks are vulnerable, though classical models can also be attacked with tailored methods.

Category: Opinion

How do you evaluate robustness to Evasion Attacks?

Use standardized threat models, adversarial benchmarks, attack suites, and report metrics like robust accuracy under varying perturbation budgets.

Category: Testing

What is currently a hot topic around Evasion Attacks?

Certified robustness, adaptive attacks against defenses, and robust training at scale are active research areas.

Category: Topic

Where to start learning about Evasion Attacks?

Read foundational papers on adversarial examples, explore MITRE ATLAS, and practice with toolkits like CleverHans or Foolbox.

Category: Example