Loading...
Loading...
Control request rates to prevent abuse and ensure fair usage
Throttles and shapes API traffic via 4 industry-standard algorithms.
You already know from the API-gateway module how one front door absorbs every outside request and becomes the chokepoint, and from the horizontal-scaling module how one greedy tenant can starve every neighbor on shared machines. A single misconfigured loop fires ten thousand login attempts a second. Password guessing goes unthrottled, one scraper eats the API budget, and real users queue behind garbage. Rate limiting, which caps how many requests any key like a user, address, or token may make in a window, turns slow-site mysteries into polite explicit rejections with a 429 status (the code for slow down and retry later) plus headers saying when.
Think of a bouncer with a clicker, the one comparison we will use here: the first hundred in line enter, the rest hear the wait, and nobody inside gets crushed. In the lab above, fire a flood with no limit and watch good and bad traffic drown together, then enable a limit and watch the flood get 429s while normal users pass, because the clicker counts per key instead of blaming everyone.
The naive server treats request 1 and request 10,001 identically until it falls over. We rejected trusting clients to pace themselves because one buggy loop undoes every capacity plan. The limited server spends a tiny counter check per request to protect the expensive work behind it. That counter is the cheapest insurance in the stack.
A token bucket holds up to N tokens (permissions to make one request each) and refills at R per second. Each request spends one token, and empty means reject. With capacity 10 and refill 1 per second, an idle client bursts 10 instantly, then sustains 1 per second: after a 10-request burst at second 0, second 1 has 1 token, second 2 has 1 more. In the lab, idle then burst and watch the bucket drain from 10 to 0 before the steady drip takes over, because savings spend fast and earnings drip slow.
A leaky bucket queues arrivals and releases them at a fixed drip, say 1 per second with room for 5 waiting. A 10-request burst fills the 5 slots and rejects 5 immediately, then drains one per second for 5 seconds. Bursty callers get rejects while steady callers sail through, which is exactly the smoothing video players want: the network jitters but playback sees a calm drip. The edge case is latency: queued requests wait, so realtime calls prefer token buckets that reject instead of delaying.
Fixed windows count requests inside calendar slices like 09:00 to 09:01 with a cap of 100. Simple, tiny state, and broken at edges: 100 requests at 09:00:59 plus 100 at 09:01:00 means 200 requests inside 2 seconds while each window innocently reports 100. That boundary spike is the whole flaw. In the lab, aim bursts at the boundary and watch the 2-second rate double the advertised limit, because the counter forgets every 60 seconds.
100 at the end of one window plus 100 at the start of the next equals 200 in about 2 seconds against a 100-per-minute promise. Use only where brief doubles cannot hurt.
A sliding log stores each request's timestamp and counts only the last 60 seconds on every arrival. A new request at 09:31:30 drops everything before 09:30:30, counts survivors, and rejects at 5. Exact, no boundary trick, and hungry: 5,000 requests per minute per key means 5,000 timestamps per key. The one-layer-deeper fix is a sliding counter that blends the last two fixed windows by overlap instead of storing every stamp, keeping most of the accuracy for a fraction of the memory.
| Method | Memory | Exactness | Bursts |
|---|---|---|---|
| Token bucket | Tiny | Good | Welcomed |
| Leaky bucket | Tiny | Good | Smoothed |
| Fixed window | Tiny | Weak at edges | Doubles possible |
| Sliding log | Large | Exact | Blocked |
Rule of thumb: token buckets for user-facing APIs that should burst, leaky buckets for pipelines that must drip, sliding counters for paid quotas where doubles cost money. Whatever you pick, limit per key across all servers (a shared counter store, not per-box memory), or ten boxes each allow 100 and the real limit becomes 1,000.
Payment processors, services that move money per API call and must survive buggy loops, cap reads and writes separately per key and return remaining-quota headers, so one runaway script gets 429s instead of a five-figure bill.
Code-hosting APIs, services letting scripts query code data that scrapers love, allow thousands of calls per hour per user on sliding windows, precisely to erase the boundary double-spend fixed windows would grant.
Buckets for bursts, drips for smoothness, and a polite 429 instead of a mysterious timeout. Throttling keeps the peace at the door, but some failures come from inside: a dependency dies mid-call, and every waiter piles onto the corpse. What refuses those doomed inside calls fast enough that the caller itself survives?
Try this in the playground
Open a template and build it yourself — then take a quiz.