🕳️Backdoor Attack

NEW

Also known as: Trojan Attack

Training-time attack that implants hidden triggers so models misbehave on targeted inputs while appearing normal otherwise.

DESCRIPTION

A backdoor attack involves poisoning training data or the training process to embed a trigger that activates malicious behavior. When the trigger appears in an input, the model outputs an attacker-chosen result; without it, performance looks normal. These attacks threaten ML supply chains, outsourced training, and model marketplaces. Defenses include data sanitization, spectral signature analysis, robust training, and trigger detection. Model monitoring can detect anomalous patterns post-deployment. Secure pipelines, provenance tracking, and model vetting reduce risk. Backdoors can affect classification, OCR, speech, or LLM tool-use behaviors. They are distinct from evasion attacks, which manipulate inputs at inference without modifying training.

ALTERNATIVE DEFINITIONS

MITRE ATLAS

Backdoor attacks insert hidden behaviors during training that cause a model to produce attacker-chosen outputs when a specific trigger is present.

Focuses on training-stage hidden behaviors with triggers.

Source

Gu et al. (BadNets)

We show that neural networks can be compromised to misclassify inputs containing attacker-chosen triggers while maintaining high accuracy on clean inputs.

Seminal paper demonstrating practical neural backdoors.

Source

RELATED TERMS

Data Poisoning Adversarial Examples Supply Chain Attack Robustness Poisoned Dataset Trigger Pattern Model Hardening Responsible AI Model Monitoring

COMPETING TERMS

Evasion Attack

KNOWLEDGE AREAS

AI MLOps Data Security AI Governance Testing & Validation

TARGET AUDIENCE

AI Engineer MLOps Engineer Data Scientist Chief Technology Officer Data Governance Manager

EXTERNAL SOURCES

• Wikipedia - Data poisoning
• MITRE ATLAS - Technique profile for backdoor attacks
• arXiv - BadNets original paper

TOP QUESTIONS

What is a backdoor attack in machine learning?

It’s a training-time compromise that embeds a hidden trigger so the model outputs attacker-chosen predictions when the trigger appears, while behaving normally otherwise.

Category: Overview

How is a backdoor attack different from an evasion attack?

Backdoor attacks modify training data or process to plant behaviors; evasion attacks modify inputs at inference without changing the model’s training.

Category: Comparison

What systems are vulnerable to backdoor attacks?

Any model trained on untrusted data or infrastructure is vulnerable, including vision, speech, NLP, and LLM tool-use pipelines.

Category: Risk

How can I detect a backdoor attack?

Use data sanitization, outlier detection, spectral signatures, activation clustering, and trigger reverse-engineering. Monitor for anomalous outputs post-deployment.

Category: Detection

How do I mitigate backdoor attacks?

Secure the training supply chain, verify data provenance, use robust training, and test models against suspected triggers. Employ least-privilege and signed artifacts.

Category: Mitigation

Why are backdoor attacks hard to find?

Triggers are rare, hidden, and designed not to affect normal performance, making them difficult to detect with standard validation.

Category: Challenge

What is a real-world example of a backdoor attack?

Image classifiers misclassifying stop signs with a small sticker trigger are a classic example from research literature.

Category: Example

How should MLOps handle backdoor risks?

Implement secure data pipelines, artifact signing, model vetting, red-teaming, and continuous monitoring with drift and anomaly detection.

Category: MLOps

Are LLMs susceptible to backdoor attacks?

Yes—fine-tuning or prompt-based triggers can induce tool-use or content behaviors that activate only with specific cues.

Category: LLM