DataTerminology.comNEW
Also known as: LLM Jailbreak
Adversarial prompts crafted to bypass a model’s safety controls and elicit restricted or harmful outputs.
DESCRIPTION
A jailbreak is a targeted prompt or strategy designed to circumvent an AI model’s safety policies. It manipulates instructions or context to override guardrails and produce prohibited content. Defenses include prompt hardening, content filters, safety-tuned models, and layered runtime checks. Red teaming and monitoring help discover and mitigate emerging jailbreak techniques. Managing jailbreak risk is central to AI governance and safe deployment.
ALTERNATIVE DEFINITIONS
Microsoft Learn
Prompt injection attacks manipulate a model with crafted inputs to override its instructions and perform unintended actions.
Defines the broader attack class that includes jailbreak-style prompts.
OWASP
Prompt injection is a technique where an attacker supplies malicious input that causes an LLM to ignore previous instructions and execute the attacker’s instructions.
Security framing and taxonomy of LLM-specific risks including jailbreaks.
LLM Jailbreak Taxonomy
We provide a taxonomy of jailbreak attacks and demonstrate methods to circumvent safety alignment in large language models.
Academic characterization and categorization of jailbreak attacks.
RELATED TERMS
Prompt Injection Red Teaming Safety Filters Adversarial Prompt Guardrails Alignment Policy Violation Assistant Prompt Model Exploit
COMPETING TERMS
Content Safety
KNOWLEDGE AREAS
AI Governance AI Data Security Testing & Validation
TARGET AUDIENCE
AI Engineer MLOps Engineer Data Scientist Data Governance Manager Chief Data Officer AI Product Manager
EXTERNAL SOURCES
- Wikipedia - Prompt injection
- Microsoft - Security guidance on prompt attacks
- OWASP - LLM security risk taxonomy
- Google - Policy and safety guidance
TOP QUESTIONS
Why are Jailbreaks a concern for AI systems?
They can force models to output harmful, confidential, or noncompliant content, creating safety, legal, and reputational risks.
Category: Risks
How do Jailbreaks differ from prompt injection?
Jailbreaks are a subset focused on bypassing safety constraints; prompt injection encompasses broader instruction-overriding attacks.
Category: Comparison
What defenses mitigate Jailbreaks?
Layered defenses: robust system prompts, safety-tuned models, content filters, rate-limits, and monitoring with red teaming.
Category: Mitigation
What are common mistakes in preventing Jailbreaks?
Relying only on prompt engineering, or not testing against evolving adversarial techniques.
Category: Mistakes
How do you test for Jailbreak vulnerabilities?
Conduct red teaming with known jailbreak prompts, adversarial generation, and evaluate refusal behavior.
Category: Testing
Do fine-tuned models resist Jailbreaks better?
Safety-tuned models often resist better, but no solution is foolproof; continuous updates are required.
Category: Opinion
What telemetry helps detect Jailbreak attempts?
Track refusal rates, policy violation categories, unusual token usage, and repeated adversarial phrasing.
Category: Observability
Can guardrails eliminate Jailbreaks completely?
No. They reduce risk but attackers continually adapt, necessitating ongoing defense-in-depth.
Category: Misunderstanding
How does Jailbreak risk relate to regulation?
Regulators may expect safeguards and incident response; noncompliance can trigger penalties or mandatory reporting.
Category: Compliance