Loading...
Loading...
Asynchronous communication between services
You already know from the synchronous-RPC module how holding a connection open through slow work burns threads, and from the back-pressure module how an unbounded arrival spike needs somewhere explicit to wait. A user uploads a video and waits while your server transcodes it before responding. Thirty seconds of staring. Now multiply by ten thousand uploads an hour. The fix is not faster transcoding, it is answering before the work is done, and doing the work where the user cannot see it. A queue (a durable pile of task notes that senders drop and workers chew through independently) is that pile.
Think of a deli ticket counter, the one comparison we will use here: you take a number in a second and sit down while the kitchen works through tickets at its own pace, calling numbers when ready. In the lab above, spike arrivals with no pile and watch latency and errors climb together, then add the pile and watch responses stay instant while the pile depth absorbs the rush, because waiting moved from the user to the ticket rail.
The naive server finishes each job inside the request: hold the connection open for the full ten-minute encode, and a crash loses both the work and the user. We rejected scaling that layout with more request threads because thread count only postpones the same collapse. The queued server writes a note and replies at once, so crashes only delay a note that still sits safely on disk. Run the numbers: 100 uploads per minute at 10 minutes each needs 1,000 simultaneous workers without a pile, but any worker count survives with one, at the price of waiting.
Producers (programs that create tasks, like the upload handler) fire tickets without waiting. The broker (the service holding the pile durably) keeps them through crashes and rush hours. Consumers (worker programs that take tickets and do the slow work) chew at their own pace. Nobody calls anyone directly, so each side scales alone. The pile itself is the coordination: depth signals add workers, emptiness signals scale down.
In the lab, raise senders with workers fixed and watch depth grow while replies stay flat, then add workers and watch depth drain at roughly workers times speed, because throughput is mouths times appetite.
A ticket is not deleted when handed out, only when the worker confirms completion with an ack (a short acknowledgment message meaning done, delete it). Crash mid-bite without an ack and the ticket reappears after a visibility timeout (a grace period before redelivery) for the next worker. That hand-out, confirm, then delete rule is the entire reliability story. The edge case is a poison ticket that always crashes its worker: cap redeliveries and park it in a dead-letter pile (a side queue for tickets that failed too often) for humans instead of looping forever.
The pile does not care how many workers feed from it. Two workers drain about twice as fast, ten about ten times, until the shared database or downstream they all call becomes the new ceiling. That is the nuance: queues scale the doing, not the dependencies of the doing. In the lab, double workers under a downstream limit and watch depth stop shrinking at the downstream's rate, because the slowest shared dependency sets the pace.
Queues delete after one worker finishes: each ticket goes to exactly one mouth, which fits jobs like emails, encodes, and charges. Logs or streams (append-only histories that keep every event and let each reader replay at its own pace) fit observations like clicks and audits, where analytics, fraud, and recommendations all read the same past independently. The one-layer-deeper difference is retention: queues forget on ack while streams remember by time or size, trading storage for replay.
Tools like RabbitMQ (a classic broker you run) and SQS (a managed queue service from a cloud provider) delete on ack. Perfect for work that must happen once-ish: receipts, encodes, charges.
Tools like Kafka (a distributed log you run) and Kinesis (its managed cloud cousin) keep events for days and track each reader's position. Perfect for facts many teams consume: analytics, audits, feeds.
Your ride receipt arriving a minute after the trip is a queue doing its job: the trip reply was instant while charging, emailing, and fraud checks finished as tickets.
Every click stored once and replayed by many teams is a stream paying off: one durable past feeds recommendations today and fraud models retrained next month.
Piles turn wait-for-it into we-will-handle-it. But a pile of promises raises the next question: what exactly was promised, at most once, at least once, or exactly once, meaning delivered zero-or-one, one-or-more, or precisely-one time. If your workers can charge a card twice by accident, which of those three guarantees would you actually pay for?
Try this in the playground
Open a template and build it yourself — then take a quiz.