Module 33 Distributed Coordination Mastery Capstone
Masterless distributed databases (Dynamo, Cassandra) survive catastrophic node outages by pairing quorum replication ($W + R > N$) with vector clock metadata. Strong consistency emerges probabilistically when read and write quorums overlap 🌐👑.
class QuorumReplica:
def __init__(self, node_id, is_alive=True):
self.node_id = node_id
self.is_alive = is_alive
self.store = {}
def write(self, key, value, vclock):
if not self.is_alive:
return False
self.store[key] = (value, vclock)
return True
class QuorumCoordinator:
def __init__(self, replicas, write_quorum=2):
self.replicas = replicas
self.write_quorum = write_quorum
def put(self, key, value, client_id):
vclock = {client_id: 1}
acks = sum(r.write(key, value, vclock) for r in self.replicas)
return acks >= self.write_quorum
nodes = [QuorumReplica("A"), QuorumReplica("B"), QuorumReplica("C", is_alive=False)]
coord = QuorumCoordinator(nodes, write_quorum=2)
success = coord.put("theme", "dracula", "client_1")
print(f"Quorum write satisfied (2/3): {success} 🚀")
# 45. Module 33 Distributed Coordination Mastery Capstone Building distributed systems without consensus and quorum math is just gambling with user data. When network cables get severed or AWS availability zones fail, only mathematically rigorous coordination primitives keep your cluster intact 🛑. ```python # The Quorum Invariant: # N = 3 (Total Replicas) # W = 2 (Write Quorum) # R = 2 (Read Quorum) # W + R = 4 > 3 -> Guaranteed Overlap! ``` ### The Production Masterless Formula 1. **Consistent Ring Placement**: Keys hash to virtual positions, determining the primary replica range without central routing masters 🌐. 2. **Sloppy Quorums & Hinted Handoff**: If Node C is down during write, the coordinator deposits the entry on Node D with a hint to forward it to Node C when it recovers. 3. **Read Repair**: If a read quorum reveals divergent versions, the coordinator uses vector clocks to determine the newer version, serves it to the client, and asynchronously writes it back to stale nodes 🛠️. --- ### Interactive Pop-Quiz 🧠 #### According to the distributed quorum formula $W + R > N$, why does overlapping write ($W$) and read ($R$) quorums guarantee strong consistency? - [ ] Because the set of nodes read from is mathematically guaranteed to intersect with the set of nodes written to by at least one replica 🌐 - [ ] Because it forces the Linux network kernel to prioritize TCP socket packets - [ ] Because CPU instruction caches are automatically flushed upon quorum confirmation - [ ] Because it prevents hard drive read heads from vibrating out of alignment --- ### Practical Challenge 🎯 Implement `QuorumCluster` with 3 nodes. Node C has `alive=False`. Define `put(key, value, client_id)` requiring write quorum $W=2$ and `get(key)` requiring read quorum $R=2$. Write key `'balance'` with value 500. Read it back, verifying the latest vector clock value is returned. Print `'QUORUM_OK'`. <details> <summary>Solution</summary> ```python class Node: def __init__(self, nid, alive=True): self.nid = nid self.alive = alive self.data = {} def write(self, k, v, vclock): if not self.alive: return False self.data[k] = (v, vclock) return True def read(self, k): if not self.alive: return None return self.data.get(k) class QuorumCluster: def __init__(self, nodes, w=2, r=2): self.nodes = nodes self.w = w self.r = r def put(self, k, v, cid): vclock = {cid: 1} acks = sum(1 for n in self.nodes if n.write(k, v, vclock)) return acks >= self.w def get(self, k): responses = [n.read(k) for n in self.nodes if n.read(k) is not None] if len(responses) >= self.r: return responses[0][0] return None nodes = [Node("A"), Node("B"), Node("C", alive=False)] cluster = QuorumCluster(nodes, w=2, r=2) assert cluster.put("balance", 500, "client_1") val = cluster.get("balance") assert val == 500 print("QUORUM_OK") ``` </details>