Loading...
Loading...
Consistency, Availability, Partition tolerance - pick two
You left foundations with three copies holding a $50 gift-card balance, a quorum knob set to two of three, and staleness priced at milliseconds in-region or hundreds of milliseconds across an ocean. Now make it concrete: your database runs in two AWS regions, Virginia on the US east coast and Frankfurt in Germany. The Atlantic link between them starts dropping half its packets. Writes land in Virginia. Frankfurt keeps serving reads of what, exactly? Data from before the storm, or nothing at all?
Eric Brewer, a computer scientist who stated the tradeoff, named this trap in 2000: CAP, where C means consistency, A means availability, and P means partition tolerance. Three properties, two survivors. And the cruel part is not the choosing. It is that the network chooses when you must choose, usually at 3am.
Consistency means every read sees the latest write or fails trying. Ten servers, one shared reality. The moment they disagree, someone gets an error instead of a lie, because the system refuses to present a stale copy as truth.
Availability means every request gets a response, possibly last minute's truth, but never silence. The lights stay on even when the servers cannot hear each other, because each side answers from what it knows.
Partition tolerance means keeping working when the network between your own machines breaks. A partition is that split itself. It is not optional in practice, because cables, switches, and entire regions fail on schedule. You do not opt out of P. You only pick what it costs you.
Walk through the split second by second. Virginia accepts a write. Frankfurt cannot hear Virginia because the Atlantic link between the two regions is dropping packets. A reader asks Frankfurt for that value. Two futures exist and both hurt somebody, because no message can cross the broken link to settle it.
Refuse to answer until the network heals. Truth preserved, user staring at a spinner. That is CP, meaning consistent and partition-tolerant: correctness survives the storm, responsiveness does not.
Answer from memory and reconcile later. User happy now, possibly wrong, with your code owning the merge when Frankfurt and Virginia compare notes. That is AP, meaning available and partition-tolerant: responsiveness survives, perfect agreement waits.
“Why not CA, consistent and available, just skip partitions?” Because skipping partitions means assuming the network never fails. Every team that assumed that has a war story about the night it did. The rejected CA design dies on one number: a single 90-second partition a month already exceeds the downtime budget the design pretended was zero. Partitions are not a risk to manage. They are weather. Single-machine systems can pretend otherwise for a while, while anything spanning racks, zones, or regions cannot.
CP systems would rather error than lie. During a split, affected requests get timeouts while the system waits for a quorum it can trust. A quorum here means a majority of copies agreeing, so no two majorities can disagree about the same write.
The bill: correctness you can audit, paid for in availability. During the storm, some users get nothing rather than something wrong. What breaks first is write availability on the minority side: with five voters and only two reachable, no majority exists, so writes stop even though the machines are healthy.
AP systems keep serving from whatever each side knows, then merge when the network heals. The name for that promise is eventual consistency, which means that if writes pause, all copies converge to the same value. Keep writing and they chase forever, which is fine because that is what a live system does. “Eventually” in one region usually means milliseconds to seconds; across Virginia and Frankfurt it means at least the 80–100ms the ocean demands, plus queueing.
The bill: availability you can feel, paid for in reconciliation logic and the occasional “wait, that is not what I ordered” moment. The edge case that bites is the double effect: both sides accept a charge or a seat booking, and the healed system discovers it sold one thing twice.
| Who | Picks | The one-line reason |
|---|---|---|
| Shopping carts on DynamoDB, Amazon's hosted store | AP | Never refuse to take money; merge the cart later |
| Spanner, Google's global database | CP | Global money movement with tight clock coordination backing the promise |
| Inbox and feed stores on Cassandra-style tech | AP | Search must answer even mid-storm; staleness is invisible here |
| Core banking | CP | A wrong balance is a lawsuit; an error page is an apology |
CAP only speaks during splits, which, on a healthy day, is almost never. The rest of the time a subtler trade runs every request: latency versus consistency. Engineers call the fuller picture PACELC, which adds “else” to CAP: if partitioned, choose availability or consistency, else choose latency or consistency. A cross-region synchronous write, meaning Virginia waits for Frankfurt before answering, is consistent on a sunny day and still slow, because physics charges over a hundred milliseconds per round trip whether or not anything failed.
| Design | Sunny-day write cost | Storm behavior |
|---|---|---|
| Single-region quorum (3 nodes) | ~1–5 ms added per write | Loses writes if quorum unreachable (CP) |
| Cross-region synchronous | +100–200 ms per write, always | Consistent but unavailable on split (CP) |
| Local write, async replicate | ~1 ms, replies immediately | Keeps answering, reconciles later (AP) |
Spanner, Google's global database, is the famous attempt to soften this with TrueTime, which is Google's system of tightly synchronized clocks that lets servers agree on ordering with single-digit-millisecond coordination in the common case. The lesson is not to copy it but to price it: consistency always has a latency cost, split or no split, and the table above is the receipt.
“CP or AP” sounds binary until you meet quorum math. With N copies, pick a read quorum R, meaning how many copies a read consults, and a write quorum W, meaning how many copies must confirm a write. If R plus W is greater than N, every read overlaps at least one copy that saw the latest write and sees fresh data, which is strong consistency, meaning every read sees the newest write, paid in latency and fragility. Walk it with N equals 3: W equals 2 and R equals 2 overlap, since 2 plus 2 is 4, which beats 3, so reads are fresh but every write waits on two confirmations. Drop to W equals 1 and R equals 1 and nothing overlaps, so writes fly and reads can be stale. Stores in this style let operators slide R and W per query: W equals 1 for a fast cart add, R plus W greater than N for the checkout that must be right.
That per-operation dial is the working answer: checkout stricter than browsing, balances stricter than feeds. And the sunny-day half still applies with no partition in sight, because every extra replica you wait on adds its round trip to each write, so strong consistency costs latency on every write, storm or shine.
You can now slide R and W per operation: W equals 1 for the 1 ms cart add, R plus W greater than N for the checkout that waits on two confirmations. But that dial only promises convergence, not timing. When a user changes their name and reloads in the next second, which notch of the dial decides whether they see the old name, and for how long?