Loading...
Loading...
The last topic left you with swappable boxes sized by arrival rate times handling time, guarded by a 60–70% CPU trigger and 30–120 second slow-start ramps. But right now every one of those boxes shakes hands with strangers directly: they learn real IP addresses, negotiate encryption with app code, and hammer endpoints that were never designed for the open internet. One misbehaving client, one leaked address, and the damage lands directly on the machine doing your business logic.
The fix is a gatekeeper at the door: all strangers talk to it, it talks to your servers. Users never learn your real addresses. That gatekeeper is a reverse proxy, “reverse” because it stands in front of servers, where a forward proxy, the kind a company laptop uses to reach the internet, stands in front of clients.
One line to remember: forward proxies hide clients from servers; reverse proxies hide servers from clients.
Here’s why nearly every production site runs one even with a single server behind it, each job below is something your app code would otherwise do badly:
Cousins, not twins. A load balancer’s whole job is spreading traffic. A reverse proxy handles requests, and happens to balance as one of six jobs:
| Question | Reverse proxy | Pure load balancer |
|---|---|---|
| Useful with one server? | Yes, security, TLS, caching all still apply | Pointless, nothing to spread across |
| Caches responses? | Yes | Usually no |
| Terminates TLS? | Yes | Sometimes |
In practice: NGINX, an open-source web server often used as a proxy, and HAProxy, an open-source load balancer, do both jobs, which is why the terms blur. Reach for NGINX first; specialize later.
Encryption is the quiet CPU fire your app servers never signed up for. A single TLS handshake involves asymmetric crypto thousands of times heavier than serving a cached byte, and at a few thousand new connections per second it becomes a visible slice of fleet CPU. Terminating TLS once at the proxy and reusing keepalive connections upstream means backends handle plain HTTP over warm, persistent sockets, Cloudflare, an edge network operator, and every major CDN, a network of cache servers stationed near users, run exactly this shape, and it is why a lone NGINX box in front of a single app server still earns its keep.
| Knob | Sane starting value | Why |
|---|---|---|
| Backend keepalive pool | 32–100 idle connections per backend, 60s idle timeout | Kills per-request handshake churn; too many idles waste backend sockets |
| Client body size limit | 1–10MB default, raise per-route for uploads | Unbounded bodies let one uploader pin proxy memory |
| Proxy timeouts | Connect 5s, read/send 30–60s | Slow-connection attacks that drip bytes to hold workers forever without them |
| Compression threshold | Compress text over ~1KB with gzip/Brotli, never images | Compressing tiny or already-compressed bodies wastes CPU for bytes |
Before installing a gatekeeper, teams terminate encryption in each app server and expose them directly behind firewall rules. It fails on arithmetic and operations. A single TLS handshake, the initial cryptographic negotiation before any request byte moves, involves asymmetric crypto thousands of times heavier than serving a cached byte, so a few thousand new connections per second becomes a visible slice of fleet CPU on every backend instead of one proxy tier. Worse, every certificate renewal, cipher-policy change, and handshake optimization touches every backend in every language instead of one config. Central termination pays the crypto bill once and renews in one place, which is why the per-app approach always consolidates back to the door.
A reverse proxy that trusts headers becomes a confused deputy. The backend readsX-Forwarded-For, the header proxies use to record the original client address, to rate-limit “users”, but any client can forge that header, so the attacker gets a fresh identity per request while real users sharing the proxy’s IP get blocked together. The fix is unglamorous: the proxy overwrites the client-supplied header with the real peer address, and only headers from trusted upstream hops (your CDN, your mesh) are honored.
Caching at the proxy adds a second lie vector: cache poisoning via an unkeyed header. If the cache key is path-only but the response varies on a header the key ignores, one attacker’s crafted request gets stored and served to everyone after. Users see someone else’s content or a broken page, and purging feels like whack-a-mole because the poison re-caches on the next crafted hit. Key on everything the response varies on, or do not cache the route at all.
A load balancer spreads traffic; a reverse proxy represents your servers, it terminates encryption, filters, compresses, caches, and happens to balance as one of its jobs, which is why it pays off even with a single backend. Every job consolidated at the edge is a single place to fix and a single place to fail, so keep request-shaping at the proxy and business logic strictly behind it. That is also why teams rarely let app servers terminate encryption themselves: then every deploy, rotation, and cipher upgrade touches every backend, private keys sprawl across the fleet, and each application language implements encryption slightly differently. Central termination means one renewal path, one cipher policy, one handshake-optimization target, and backends that stay simple, fast, and replaceable.
Your front door is now guarded, encrypted, cached, and balanced, with TLS terminated once, a 32–100 connection keepalive pool of long-lived reused upstream sockets, and forged X-Forwarded-For headers overwritten at the edge. But everything behind that door is still one codebase, one deploy, one blast radius: the payments team ships a fraud check and the menu code they never read goes down with it. The open question is what happens when the app behind the proxy outgrows a single deployable, and how you cut it without cutting the reliability you just built.