Understanding the CAP Theorem in Practice

What CAP Actually Says

Eric Brewer conjectured in 2000 — and Gilbert and Lynch formally proved in 2002 — that a distributed data store cannot simultaneously guarantee all three of the following properties:

• Consistency — every read returns the most recent write. In CAP, this specifically means linearizability: the system behaves as if there is a single copy of the data, and every operation takes effect atomically at some point between its start and its completion. This is not the same "C" as in ACID transactions, which refers to maintaining database invariants (foreign keys, constraints). CAP consistency is about read freshness across nodes.
• Availability — every request to a non-failing node receives a response (not an error or a timeout). The response doesn't have to be the latest data — it just has to be a valid, non-error response.
• Partition Tolerance — the system continues to function when messages between nodes are lost or delayed. A partition is a communication failure, not a node crash. Two nodes are both up and running, but they cannot talk to each other.

Why "Pick Two" is Misleading

The original framing — "pick two out of three" — suggests you have three equal options: CA, CP, or AP. This framing causes confusion because it implies you can opt out of partition tolerance.

You cannot. If your system runs on more than one machine connected by a network, network partitions will happen. Switches fail, cables get cut, cloud availability zones lose connectivity to each other. These are physical events, not configuration options. A system that cannot handle a partition is a system that goes down during a partition — which means it has already sacrificed availability.

This is why Martin Kleppmann and Brewer himself have clarified the theorem over the years: the real question is not "which two do you pick?" The question is: when a partition occurs, does your system sacrifice consistency or availability?

CP: Choosing Consistency During a Partition

A CP system, when it detects that nodes cannot communicate, stops accepting operations that could produce inconsistent state. In practice, this usually means the minority side of the partition either rejects writes, rejects reads, or becomes entirely unavailable until the partition heals.

Consider a three-node cluster using a consensus protocol like Raft. If a network partition splits the cluster into a group of two and a group of one, the group of two still has a majority quorum. It can continue accepting reads and writes because it can guarantee consistency among a majority of nodes. The isolated node, unable to reach a quorum, stops serving requests.

Network partition splits a 3-node Raft cluster:

 [Node A] ◄──► [Node B]     ✗ network cut ✗     [Node C]
       (majority quorum)                         (no quorum)
     
 Node A + B: continue serving                Node C: rejects requests
 reads/writes (leader is here)               (returns error or timeout)

The system remains consistent — every successful read reflects the latest write — but Node C is unavailable for the duration of the partition.

Where CP makes sense

Financial systems, inventory management, distributed locks — anywhere stale data causes concrete damage. If you're reserving the last seat on a flight, two nodes independently confirming that seat is available is worse than one of them returning an error.

Examples: ZooKeeper, etcd (both Raft-based), HBase, MongoDB with majority read/write concern, Google Spanner.

AP: Choosing Availability During a Partition

An AP system keeps every node responsive, even when they cannot synchronize with each other. Both sides of a partition continue accepting reads and writes. The consequence is that different nodes may have different versions of the same data.

When the partition heals, the system must resolve the conflicts. There are several strategies for this, none of them simple:

• Last-write-wins (LWW) — the write with the highest timestamp survives. Simple, but silently discards data. If two users edit the same record during a partition, one edit disappears.
• Vector clocks — each node maintains a version vector tracking which writes it has seen. When conflicting versions are detected, the system can surface the conflict to the application for resolution (DynamoDB does this).
• CRDTs (Conflict-free Replicated Data Types) — data structures designed so that concurrent modifications can always be merged automatically without conflicts. Counters, sets, and registers can be implemented as CRDTs.

Where AP makes sense

Systems where uptime matters more than instant consistency: social media feeds, shopping carts, activity logs, metrics collection. If a user adds an item to their cart and a different node doesn't see it for 200 milliseconds, no real harm is done. If the entire cart service goes down during a partition, that's lost revenue.

Examples: Cassandra, DynamoDB (default eventually consistent mode), CouchDB, Riak.

AP system during a network partition:

  User A writes price=$10 ──► [Node 1]  ✗ partition ✗  [Node 2] ◄── User B writes price=$12

  Both writes succeed. Both nodes return HTTP 200.

  After partition heals:
  Node 1 has: price=$10
  Node 2 has: price=$12
  
  Conflict resolution required:
    LWW    → higher timestamp wins (one write is silently lost)
    Vector → application decides which value to keep
    CRDT   → only works for specific data types (counters, sets)

Beyond CAP: The PACELC Model

CAP only describes what happens during a partition. But partitions are rare events. Most of the time, your distributed system is running normally with all network links healthy. During normal operation, you still face a tradeoff — and CAP has nothing to say about it.

Daniel Abadi proposed the PACELC theorem in 2012 to fill this gap. It reads:

If there is a Partition (P), choose between Availability (A) and Consistency (C); Else (E), choose between Latency (L) and Consistency (C).

The "Else" clause captures a fundamental reality: maintaining consistency across multiple nodes requires coordination. A strongly consistent read might need to contact a quorum of replicas and wait for confirmation before returning. That round trip adds latency. An eventually consistent read can return immediately from whichever replica is closest, but the data might be slightly stale.

This tradeoff exists every day, on every request, not just during network failures. PACELC makes it explicit.

Real systems through the PACELC lens

Notice that some systems are configurable. DynamoDB lets you request strongly consistent reads on a per-request basis. Cassandra lets you configure the consistency level per query (ONE, QUORUM, ALL). The PACELC classification describes the default behavior, but many systems let you shift along the spectrum per operation.

The Part That CAP Doesn't Cover

CAP is useful as a mental model, but it has real limitations. It treats consistency and availability as binary — you either have linearizability or you don't, you either respond to every request or you don't. Real systems operate on a spectrum.

Between linearizability and total inconsistency, there are many useful consistency levels: causal consistency, read-your-writes, monotonic reads, session consistency. A system might sacrifice linearizability but still guarantee that you always see your own writes — which is good enough for most applications.

Similarly, "availability" in CAP means every non-failing node responds. But practical availability is about percentages and latency targets. A system with 99.99% uptime and a 10ms P99 is more useful than a system that technically responds to every request but sometimes takes 30 seconds.

The takeaway: use CAP to understand the fundamental constraint. Use PACELC to think about the everyday latency-consistency tradeoff. Use neither as a checklist — use them as starting points for asking the right questions about your system's requirements.