Loading...
Loading...
Distribute incoming requests across multiple servers efficiently
Distributes traffic concurrently. Evaluates nodes based on strategies, queues spikes (max 4 active), and skips offline servers.
You already know from the horizontal-scaling module how adding identical servers raises capacity, and from the DNS module why handing out raw addresses leaves clients stuck on dead machines. With three servers and no balancer, the answer is your users. Play with the sim above, kill a backend mid-rush and watch what happens with nobody routing around it. Then flip the balancer on. A load balancer, the one address clients know that forwards each request to a healthy server behind it, quietly sends each request to a server that is actually alive.
Think of the host at a restaurant door, the one comparison we will use here: diners queue at one stand and the host seats each party at a table with room, skipping the table whose chairs just broke. Health checks (small repeated probes like GET /health that mark a server down when they fail) are how the host learns which tables are broken. In the lab, break one backend and watch the balancer's probe fail, the server leave the rotation, and traffic continue with one fewer seat, because detection plus removal beats hoping.
The job, precisely: every request arrives at one door, the balancer picks a healthy server behind it, and stops picking servers that went dark.
The naive setup publishes every server address and lets each client pick. We rejected leaving the choice with clients because clients pick badly: they cache one address, hammer it, and keep calling it after it dies. The balancer replaces N addresses with one name plus a brain, and the brain plus probes is the whole win.
There is no best rule, only the rule whose bet matches your rush hour. Try each in the sim under even load, then under one slow server, then under sticky logins, and watch the winner change, because the traffic changed, not the math.
Round robin (dealing requests 1-2-3-1-2-3 in fixed rotation) assumes every server and every request costs the same. With 9 equal requests over 3 servers each takes exactly 3, so the arithmetic is perfect. The edge case is unequal cost: one slow server still receives every third request and its queue grows while fast servers idle. In the lab, make one backend twice as slow and watch its latency climb under turn-taking, because fairness of count is not fairness of work.
Weighted round robin (the same rotation with shares, so a weight-2 server takes twice the turns of a weight-1 server) bets that size predicts speed. A 4GB, 4GB, 8GB fleet with weights 1-1-2 deals 25%-25%-50%: over 100 requests the big box takes 50. That beats even dealing when boxes differ. The edge case is live load: weights are static labels, so a big box stuck on garbage collection still receives half the traffic. Reweigh on deploys, not on emergencies.
Least connections (always picking the server with the fewest active requests) bets that busyness predicts speed. With counts 45, 23, 38 the next request takes the 23, which adapts instantly when requests vary in length. The edge case is stampeding the freed server: fifty arrivals in one millisecond can all read 23 before any counter updates, so real balancers add small randomization or pick among the two least busy. In the lab, fire a burst at uneven request times and watch this rule beat turn-taking, because it measures instead of assuming.
Hash-based routing (hashing the visitor id or address to pick a server, so the same input always lands together) bets that memory locality beats balance. Sticky sessions (returning one visitor to the same server so in-memory login state and carts survive) make stateful apps work without shared storage. The edge case is membership change: adding a server rehashes visitors and their local state vanishes, which is why sticky hashing pairs with consistent hashing or shared session stores in serious setups.
| Rule | Effort | Fits best | Sees live load |
|---|---|---|---|
| Take turns | Low | Equal boxes, equal jobs | No |
| Proportional shares | Low | Mixed box sizes | Partly |
| Least busy | Medium | Jobs with uneven length | Yes |
| Sticky hash | Low | Logins kept in memory | No |
Cloud load balancing (a managed service that spreads traffic across your machines and zones for you) removes the door as a chore: it probes, scales itself, and absorbs zone failures. You trade control and per-hour cost for never waking up to a dead balancer.
High-performance proxies (programs like NGINX and HAProxy that you run to route and terminate connections) serve hundreds of millions of sites with every rule above plus connection encryption handling. You gain tuning and lose sleep, because now the door itself needs its own redundancy.
Global balancers answer the name directory with the nearest healthy city, then local balancers spread inside it. Two layers, two wins: geography first, busyness second.
The best probes mimic real users: fetch a real path, check dependencies, fail on slowness, not just survival. A server answering 200 OK in 9 seconds is down for every purpose that matters. In the lab, switch a probe from survival to a latency budget and watch a zombie server leave the rotation.
Balancers decide who answers. But an answer is only as honest as the data behind it, and that data lives in databases with their own scaling story. When spreading load across machines meets rows that must stay consistent, which half of the database trade-off would you sacrifice first?
Try this in the playground
Open a template and build it yourself — then take a quiz.