CAP Theorem in Practice: What You're Actually Choosing

The theorem in one sentence

A distributed system cannot simultaneously guarantee Consistency (every read returns the most recent write), Availability (every request receives a response), and Partition Tolerance (the system continues operating when network messages are dropped).

Why partition tolerance is not optional

A network partition — where some nodes cannot reach others — is not a theoretical edge case. Networks drop packets. Data centers lose connectivity. Kubernetes pods fail health checks. If your system spans more than one process, you must assume partitions will happen.

This means you do not actually choose from three options. You choose CP or AP — what to sacrifice when a partition occurs.

CP: choose consistency

When a partition happens, a CP system refuses to serve stale data. It returns an error or blocks until the partition heals and the nodes can agree on the current state.

Example: ZooKeeper, etcd. These are used for distributed coordination (locks, leader election, configuration) where a stale read would be worse than no read. If a ZooKeeper node cannot confirm quorum, it stops responding rather than returning potentially outdated state.

When to choose CP: configuration stores, financial ledgers, inventory counts, any domain where two processes acting on stale data causes a correctness problem you cannot recover from.

AP: choose availability

When a partition happens, an AP system continues responding with whatever data it has, even if that data is stale. It sacrifices consistency for uptime.

Example: Cassandra, CouchDB, DNS. Cassandra with a replication factor of 3 and a consistency level of ONE will respond even if two replicas are unreachable — you get a response, but it might not be the latest write.

When to choose AP: user-facing reads where a slightly stale result is better than a 503 error. Product catalog pages, social media feeds, analytics dashboards — these are all more tolerable with eventual consistency than with downtime.

The spectrum is wider than two options

Real systems are not strictly CP or AP. They expose tunable consistency levels that let you make the choice per-operation.

Cassandra consistency levels:

• ALL — every replica must respond (CP behavior, maximum consistency)
• QUORUM — majority must respond (balance of consistency and availability)
• ONE — any single replica responds (AP behavior, maximum availability)

Most applications combine these: writes at QUORUM (to prevent data loss), reads at ONE or LOCAL_ONE (for low latency), accepting that very recent writes may not appear immediately to all readers.

PACELC: the fuller picture

CAP only considers the partition case. PACELC extends it: even when there is no partition (E), you still trade off latency (L) against consistency (C). A system that always requires quorum confirmation before responding is highly consistent but has higher latency than one that acknowledges writes immediately and replicates asynchronously.

Most production decisions live in PACELC territory, not CAP territory. Network partitions are rare; latency vs consistency tradeoffs happen on every request.

Practical decision checklist

Before choosing a consistency model, answer these:

1. What happens if a user reads stale data? If the answer is "they see an old price" — probably fine for AP. If the answer is "they can book a seat that is already taken" — you need CP.
2. What happens if the system is unavailable? If the answer is "revenue stops" — availability matters more. If the answer is "background jobs queue up and catch up later" — consistency matters more.
3. Can you compensate? Eventual consistency becomes acceptable if you have a reconciliation process: a nightly job, an audit log, a compensating transaction. Design for it explicitly rather than hoping conflicts will not occur.

The mistake to avoid

Treating CAP as a property of the database you chose, not the configuration you applied. Cassandra can behave as CP at ALL consistency level or AP at ONE. The choice is yours at query time — make it deliberately.