🚦Load Shedding

NEW

Intentionally drops or defers requests under overload to preserve system stability, performance, and availability.

DESCRIPTION

Load shedding is an overload protection technique where systems reject or drop lower-priority work to maintain overall health. It complements backpressure, rate limiting, and circuit breakers to prevent cascading failures. Policies may use request priority, resource budgets, or queue lengths. Proper instrumentation and SLOs guide when to shed. It is a last-resort strategy after scaling and optimization fail to meet demand.

ALTERNATIVE DEFINITIONS

Google SRE Book

Load shedding is the practice of dropping requests or reducing work in an overloaded system in order to preserve overall availability.

Defines dropping work to preserve availability during overload.

Source

Microsoft Azure Architecture Center

Under extreme load, implement throttling or load shedding to maintain system health by rejecting excess requests and prioritizing critical operations.

Emphasizes rejecting excess requests and prioritization for resiliency.

Source

RELATED TERMS

Circuit Breaker Rate Limiting Graceful Degradation Timeout Retries Queueing Autoscaling Admission Control QoS

COMPETING TERMS

Backpressure Load Balancing

KNOWLEDGE AREAS

Observability & Telemetry Software Engineering Real-time & Streaming Data Data Storage & Operations

TARGET AUDIENCE

DevOps Engineer Software Developer Solution Architect Data Engineer MLOps Engineer

EXTERNAL SOURCES

• Wikipedia - Load shedding
• Google SRE Book - SRE guidance on overload
• Microsoft Azure - Throttling pattern guidance
• Netflix TechBlog - Real-world load shedding practices

TOP QUESTIONS

When should Load Shedding be applied?

Use it when demand exceeds capacity and other mitigations (autoscaling, caching, backpressure) can’t stabilize the system.

Category: Requirements

How does Load Shedding differ from Backpressure?

Backpressure slows producers; load shedding drops or rejects work to keep the system responsive.

Category: Alternatives

What metrics guide Load Shedding decisions?

Queue lengths, latency, error rates, resource utilization, and SLOs inform thresholds and policies.

Category: Observability

What are common mistakes with Load Shedding?

Blindly dropping traffic without prioritization or client signaling can break critical flows and user experience.

Category: Mistakes

How do you prioritize during Load Shedding?

Define tiers or tokens for critical operations, preserving essential paths and degrading nonessential features.

Category: Management

How does caching interact with Load Shedding?

Caching reduces load; when hot caches miss, shedding may prevent overload and cascading failures.

Category: Integration

Can Load Shedding harm SLAs?

If misconfigured, shedding can violate SLAs; with good prioritization, it protects critical SLAs under stress.

Category: Risk

How is Load Shedding signaled to clients?

Return appropriate status codes (e.g., 429) and retry-after headers to support graceful client behavior.

Category: Operations

What governance practices support Load Shedding?

Define policies, test in chaos exercises, and document scenarios with clear runbooks and stakeholder alignment.

Category: Governance