Layer 4 vs Layer 7 Load Balancing

Your balancer can’t read. Should it learn?

The previous topic left the balancer deliberately blind: round robin over stateless backends, servers holding no per-user memory between requests, behind 5-second active checks that evict a corpse in about 10 seconds, with balancers run in pairs because the distributor is the last single point of failure. That blindness is fast, but it can’t send video uploads to the big machines, beta users to the beta build, or Europeans to Europe. The moment you want decisions based on what’s inside the request, blindness stops being a virtue.

A Layer 4 balancer, one that decides using only network addresses and ports, never looks inside. A Layer 7 balancer, one that parses the actual application request, opens every envelope: slower per request, smarter per decision. That trade, speed of ignorance versus power of knowledge, is the entire L4 vs L7 debate.

Two balancers, two pairs of eyes

What “reading the letter” buys you

Canary deploys, A/B tests, blue-green releases, all of these are just “route by what the request says.” An L4 balancer physically cannot do any of them.

Pick in ten seconds

Who reaches for which

The latency tax of opening every envelope

Numbers make the trade concrete. A Layer 4 proxy forwarding TCP segments adds roughly 0.1–0.3ms per hop, it never parses, never decrypts, never buffers a full request. A Layer 7 proxy terminating TLS, parsing HTTP, matching a routing rule, and re-encrypting upstream typically adds 0.5–2ms at p50, the median request, and considerably more at p99, the slowest 1% of requests, under CPU pressure. Throughput splits the same way: a tuned HAProxy, an open-source TCP and HTTP load balancer, or Linux IPVS, the load-balancing facility built into the Linux kernel, forwards millions of packets per second at L4, while the same hardware doing full L7 TLS termination and header inspection tops out in the low hundreds of thousands of requests per second.

TLS is the biggest single chunk of that tax. A fresh TLS 1.2 handshake costs two extra round trips before the first byte of your request moves; TLS 1.3 cuts it to one, and resumed sessions cut it near zero. That is why L7 balancers live and die by session resumption and keepalive reuse to backends: without keepalives, every user request pays a brand-new backend handshake, and your “smart routing” doubles connection churn for the whole fleet.

When smart routing turns quietly stupid

Content-aware routing fails content-aware ways. Route by a beta cookie and one misconfigured client sending that cookie on every request floods three canary boxes with 80% of production traffic. Route video uploads to the big machines and a single viral upload hour saturates exactly those machines while the general pool idles. Shard by header and an attacker varies the header per request, defeating every cache behind the balancer and turning a cheap static fleet into an origin-melting miss storm.

TLS at the edge adds its own outage genre: the certificate expires, the cipher config rejects older phones, or the firewall rule meant to block malicious database queries starts blocking last names with apostrophes. Users see browser security errors, which look like the site is hacked, not misconfigured. And buffering large bodies at L7 to inspect them means a slow uploader holds balancer memory for minutes; enough of them and the balancer runs out of memory while backends sit bored.

The shortcut teams try first: route in app code, and why it loses

Before buying an L7 balancer, teams try routing inside the app: every backend parses the Host header and path itself, then proxies the request to the right peer over a fresh connection. It fails on arithmetic. Parsing and re-routing on every hop adds roughly 1–3ms per hop instead of once at the edge, TLS certificates and cipher policy sprawl across every backend instead of living in one place, and each routing change becomes a full fleet deploy instead of a single config push. Centralizing judgment in one L7 tier pays one parsing cost per request and one renewal path per certificate, which is why the app-code mesh always consolidates back to the edge.

The standard edge shape, and where streaming fits

The shape most production systems land on is L4 at the edge for raw scale and protocol freedom with L7 behind it for judgment, for example, a network-level balancer in front of an application-level balancer, which is how the major cloud providers arrange it. The trade to remember is that every header you route on is a load dimension you now have to capacity-plan, so keep the routing key small, path prefix and hostname first, cookies and headers only when a release or tenancy model forces it. One more fit worth knowing before you commit: long-lived streaming protocols like gRPC, remote calls carried over HTTP/2 streams, and WebSockets, connections upgraded from HTTP to stay open both ways, both ride persistent connections, so the balancer must pin the whole connection to one backend with HTTP/2 support end to end rather than spreading individual messages. Balance the connection, and let the streams multiplex inside it.

L4 at the edge plus L7 behind it routes brilliantly, but both tiers assume the same thing round robin assumed: any request can land on any backend. The moment a backend remembers the user who logged in there, that assumption breaks, and no routing key, however clever, fixes servers with memories. The open question is what the servers behind the balancer are allowed to remember, because interchangeability is built, not given.