DataTerminology.comNEW
A control technique that deliberately slows or limits request processing to manage load and fairness.
DESCRIPTION
Throttling controls the pace of request processing to protect systems under load. It smooths bursts, enforces fairness, and prevents resource starvation. Unlike rate limiting caps, throttling can delay or queue requests instead of rejecting them. Implementations include queue-based shaping, token delays, or dynamic admission control based on service health. Throttling is often applied at API gateways, ingress controllers, or in service handlers. Proper settings consider latency SLOs and queue depth to avoid timeouts. Combined with backpressure and load shedding, throttling improves resilience. Observability is vital to tune thresholds and prevent cascading delays.
ALTERNATIVE DEFINITIONS
AWS
Throttling ensures that API traffic is controlled to help your backend services maintain performance and availability.
Focuses on protecting backend services by controlling API traffic.
Microsoft Learn
Throttling is the process of limiting the number of requests sent to or received by an API.
Defines throttling in API design context as limiting request rates.
RELATED TERMS
Load Shedding Circuit Breaker Quotas Quality of Service Priority Queue Timeout Bulkhead Pattern Retry with Backoff
COMPETING TERMS
KNOWLEDGE AREAS
Infrastructure Software Engineering Real-time & Streaming Data Observability & Telemetry
TARGET AUDIENCE
DevOps Engineer Cloud Architect Software Developer Data Engineer
EXTERNAL SOURCES
- Wikipedia - Traffic shaping
- Microsoft Azure Architecture Center - Design pattern for throttling
- Kubernetes - API Priority and Fairness controls
- Envoy - Local rate limit filter for shaping
TOP QUESTIONS
How is Throttling different from Rate Limiting in APIs?
Rate limiting rejects excess requests, while throttling slows or queues them. Many systems use both for fairness and protection.
Category: Misunderstanding
Where should Throttling be implemented?
At API gateways, load balancers, or in service middleware so excess load is controlled before hitting fragile backends.
Category: Management
What are risks of aggressive Throttling?
Excessive delays cause timeouts and poor UX. Queues can grow unbounded without backpressure or shedding.
Category: Risk
What signals inform adaptive Throttling?
Use CPU, queue depth, latency percentiles, and error rates to dynamically adjust admission and pacing.
Category: Requirements
What’s a hot topic in Throttling?
Adaptive, ML-driven throttling at the edge and per-tenant fairness in multi-tenant platforms are trending.
Category: Topic
How does Throttling interact with retries?
Combine with exponential backoff and jitter to avoid retry storms that worsen congestion.
Category: How-to
What patterns complement Throttling?
Backpressure, circuit breakers, bulkheads, and load shedding work together to maintain stability under stress.
Category: Overview
Can Throttling improve cost efficiency?
Yes, smoothing bursts can reduce overprovisioning and autoscaling thrash, lowering infrastructure costs.
Category: Relevancy
How to test Throttling strategies?
Use load tests with burst patterns, chaos experiments, and observe latency SLOs and error rates under varying thresholds.
Category: Testing