CAP Theorem and Consistency Models, Explained Plainly

The CAP theorem is one of the most quoted and most misunderstood ideas in distributed systems. "Pick two of three" is a catchy summary but a misleading one. This guide explains what CAP actually says, the more useful PACELC extension, and how consistency choices play out in real systems.

#Key takeaways

• CAP applies to distributed data systems during a network partition: you must choose between consistency (every read sees the latest write) and availability (every request gets a non-error response).
• Partitions aren't optional in real networks, so "CA" isn't a real choice for distributed systems. The choice is CP or AP when a partition happens.
• PACELC adds: even without partitions, you trade latency against consistency.
• Consistency is a spectrum: strong, linearizable, sequential, causal, read-your-writes, eventual.
• Choose consistency per operation, based on what goes wrong if data is stale.

#The three properties (precisely)

• Consistency (C): in CAP, this means linearizability. Every read returns the most recent write, as if there were a single copy of the data. (Not the "C" in ACID, which is about integrity constraints.)
• Availability (A): every request to a non-failing node receives a (non-error) response, without guaranteeing it reflects the latest write.
• Partition tolerance (P): the system keeps operating even when network messages between nodes are lost or delayed.

#What CAP actually says

Imagine two database replicas, one in each data center, and the link between them fails:

text   Client A ─▶ [Replica 1]   ✖ network partition ✖   [Replica 2] ◀─ Client B
               balance=100                           balance=100

Client A writes balance = 50 to Replica 1. Client B reads from Replica 2. The replicas can't talk. Replica 2 must either:

• Refuse or delay the read until it can confirm the latest value → consistent, not available (CP), or
• Return 100, possibly stale → available, not consistent (AP).

That's the whole theorem: during a partition, you can't have both linearizable consistency and full availability. When the network is healthy, you can have both.

#Why "CA" isn't really an option

A single-node database is "CA" only because it's not distributed. Once data is replicated across machines, networks will partition: switch failures, misconfigurations, cloud zone issues, garbage-collection pauses that look like partitions. So distributed systems choose how to behave when that happens.

#PACELC: the everyday trade-off

CAP only describes partition behavior. PACELC extends it:

If there's a Partition, choose Availability or Consistency; Else (normal operation), choose Latency or Consistency.

Even with a healthy network, strong consistency requires coordination: waiting for replicas to acknowledge writes, or reading from a leader. That adds latency, especially across regions (latency numbers every engineer should know). Many systems relax consistency not for partitions, but to be fast every day.

#The consistency spectrum

Many products need only read-your-writes and monotonic reads for a good user experience, which is much cheaper than global strong consistency.

#Quorums: tunable consistency

Leaderless replicated stores (in the style of Dynamo and Cassandra) use quorums. With N replicas, writes wait for W acknowledgments and reads query R replicas:

textIf R + W > N  → reads overlap with the latest write (stronger consistency)
N=3, W=2, R=2 → tolerates one node failure with overlapping quorums
N=3, W=1, R=1 → fastest, most available, eventually consistent

Quorum overlap improves consistency but doesn't by itself guarantee linearizability in every failure scenario. Details like sloppy quorums and concurrent writes matter.

#Real systems, roughly categorized

These are tendencies with default configurations; most systems are tunable:

• Single-leader relational databases (with synchronous replication or reads from the leader): prioritize consistency; during failover, writes may be briefly unavailable.
• Asynchronous read replicas: reads from replicas can be stale (replication lag). That's an AP-style trade-off on the read path (database scaling: replication and sharding).
• Consensus-based stores (etcd, ZooKeeper, distributed SQL built on Raft/Paxos): CP. They need a majority to make progress.
• Dynamo-style stores: AP-leaning by default, tunable per request.
• Caches and CDNs: favor availability and latency; staleness is bounded by TTLs (caching strategies).

#Applying it: choose per operation

Ask for each operation: what's the cost of a stale read, and what's the cost of an error?

Patterns that help:

• Route reads-after-writes to the primary (or use session consistency) for the user who just wrote.
• Use conditional writes / optimistic concurrency (version numbers) to prevent lost updates.
• Compensate instead of coordinating: accept an order and reconcile later (sagas) when strict coordination is too costly (event-driven architecture).

#Common misconceptions

• "NoSQL is AP, SQL is CP." Too simplistic. Behavior depends on configuration and deployment.
• "CAP means you can only ever have two properties." Only during partitions, and even then it's about linearizability versus total availability, not all-or-nothing.
• "Eventual consistency means inconsistent forever." Replicas converge; the questions are how quickly and what users see in the meantime.

#FAQ

#Is the CAP theorem still relevant?

Yes, as a foundation for reasoning about trade-offs in replicated systems. In practice, PACELC and specific consistency models are more useful for day-to-day design decisions.

#What does eventual consistency mean for users?

Users may briefly see slightly outdated data, such as a like count lagging by a second or a new post appearing in search a bit later. Good design hides this with read-your-writes guarantees and optimistic UI updates.

#Can a system be both highly available and strongly consistent?

When there's no partition, yes, at some latency cost. Consensus-based systems stay available as long as a majority of nodes can communicate, which covers many failure scenarios, but they sacrifice availability on the minority side of a partition.

#How does CAP relate to ACID?

They use "consistency" differently. ACID consistency means transactions preserve database invariants. CAP consistency means all nodes agree on the latest value. A distributed database can offer ACID transactions and still face CAP trade-offs across replicas.