Loading...
Loading...
A detailed comparison of bursty request handling vs smooth output queueing in distributed API gateways.
Launch Interactive System SimulatorTokens are continuously added to a fixed-capacity bucket at a constant refill rate. Incoming requests consume tokens to pass through.
APIs requiring flexibility for bursts (e.g., GraphQL endpoints, payment gateways)
Requests enter a FIFO queue and are processed ('leaked') at a constant, fixed output rate regardless of ingress rate.
Egress traffic throttling, asynchronous task processing, and strict rate enforcement
| Vector | Token Bucket | Leaky Bucket |
|---|---|---|
| Traffic Handling | Supports bursts up to bucket capacity | Strict, smooth constant output rate |
| Memory Overhead | O(1) per user (token count + timestamp) | O(N) queue space for pending requests |
| Latency Impact | Zero added latency when tokens exist | Variable queueing delay added to requests |
| Implementation Complexity | Very Low (Redis counter or Lua script) | Moderate (FIFO queue + timer/worker pool) |
-- Redis Lua Script: Atomic Token Bucket Rate Limiter
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local data = redis.call("HMGET", key, "tokens", "last_updated")
local tokens = tonumber(data[1]) or limit
local last_updated = tonumber(data[2]) or now
-- Top up tokens based on elapsed time
local delta = math.max(0, now - last_updated)
tokens = math.min(limit, tokens + delta * refill_rate)
if tokens >= 1 then
tokens = tokens - 1
redis.call("HMSET", key, "tokens", tokens, "last_updated", now)
return 1 -- Allowed
else
return 0 -- Rate limited (HTTP 429)
end// Node.js Leaky Bucket FIFO Queue Implementation
class LeakyBucket {
private queue: Array<Function> = [];
constructor(private capacity: number, private leakIntervalMs: number) {
setInterval(() => this.leak(), this.leakIntervalMs);
}
add(requestHandler: Function): boolean {
if (this.queue.length >= this.capacity) return false; // Overflow (HTTP 429)
this.queue.push(requestHandler);
return true;
}
private leak() {
const nextReq = this.queue.shift();
if (nextReq) nextReq();
}
}Stripe processes millions of API requests during Black Friday sales. Merchants generate sudden spikes in API volume during flash sales.
Stripe utilizes a multi-tiered Token Bucket algorithm at their API Edge. Token buckets allow merchants to burst legitimately during sales events while protecting internal database write capacity.
Choose Token Bucket if your API needs to support legitimate user bursts without artificial latency. Choose Leaky Bucket if your downstream microservice cannot handle spikes under any circumstances.