Resilient HTTP Connection Pooling with Keep-Alive & Circuit Breaking
Repeatedly initializing TCP/TLS handshakes wastes dozens of milliseconds per remote call. Production clients maintain persistent connection pools with HTTP Keep-Alive, guarded by Circuit Breakers that trip to `OPEN` on consecutive errors to fail-fast without overwhelming crippled downstreams 🛡️⚡🔌.
import time
class CircuitBreaker:
def __init__(self, threshold=3, cooldown=5.0):
self.state = "CLOSED" # CLOSED, OPEN, HALF_OPEN
self.threshold = threshold
self.cooldown = cooldown
self.failures = 0
self.tripped_at = 0.0
def execute(self, action):
if self.state == "OPEN":
if time.time() - self.tripped_at > self.cooldown:
self.state = "HALF_OPEN"
else:
raise RuntimeError("CircuitBreaker OPEN: Fast fail 🛑")
try:
res = action()
if self.state == "HALF_OPEN": self.state = "CLOSED"; self.failures = 0
return res
except Exception as err:
self.failures += 1
if self.failures >= self.threshold:
self.state = "OPEN"
self.tripped_at = time.time()
raise err
cb = CircuitBreaker(threshold=2)
print(f"Initial State: {cb.state} 🚀")
# 98. Resilient HTTP Connection Pooling with Keep-Alive & Circuit Breaking When a downstream payment processor or internal auth service begins timing out, the worst thing your application can do is keep hammering it with thousands of incoming client requests. Doing so exhausts local connection pools, ties up event loop worker threads, and prevents the downstream service from ever recovering 🛑. ```python # The Circuit Breaker Finite State Machine: # [CLOSED] --- (threshold consecutive failures) ---> [OPEN] # ^ | # | (cooldown expires) # | v # +------------- (trial probe succeeds) ------ [HALF_OPEN] # (trial probe fails) ---------> [OPEN] ``` ### Production Resiliency Invariants 1. **HTTP Keep-Alive & Sockets**: Always reuse authenticated TCP/TLS connections via client session pooling. Establishing new TLS handshakes adds 2–3 network round-trips ($RTT$) to every operation. 2. **Fail Fast with Zero Latency**: When the circuit is `OPEN`, reject calls immediately in 0.01ms. Do not wait for a 30-second network socket timeout before failing! 3. **Half-Open Canary Probing**: After a cooldown window, transition to `HALF_OPEN` and allow exactly one trial request through. If it succeeds, the downstream is healed; if it fails, immediately trip back to `OPEN` 🛡️⚡. --- ### Interactive Pop-Quiz 🧠 #### What is the primary architectural purpose of transitioning a Circuit Breaker into HALF_OPEN state? - [ ] It allows a single trial probe request through to verify if the downstream service has recovered before fully restoring traffic 🛡️ - [ ] It re-encrypts all database passwords - [ ] It deletes the TCP socket buffer from kernel memory - [ ] It forces all client requests to run on separate server threads --- ### Practical Challenge 🎯 Implement `CircuitBreaker` with parameters `threshold: int = 2` and `cooldown: float = 0.1`. Track `state` (`'CLOSED'`, `'OPEN'`, `'HALF_OPEN'`), `failures`, and `last_failure_time`. Provide `call(fn, *args, **kwargs)` that executes `fn`, trips on consecutive failures, raises `RuntimeError('OPEN')` when cooling down, probes on cooldown expiration, and resets on success. Verify transitions and print final state. <details> <summary>Solution</summary> ```python import time class CircuitBreaker: def __init__(self, threshold: int = 2, cooldown: float = 0.1): self.state = "CLOSED" self.threshold = threshold self.cooldown = cooldown self.failures = 0 self.last_failure_time = 0.0 def call(self, fn, *args, **kwargs): now = time.time() if self.state == "OPEN": if now - self.last_failure_time >= self.cooldown: self.state = "HALF_OPEN" else: raise RuntimeError("OPEN") try: res = fn(*args, **kwargs) if self.state == "HALF_OPEN": self.state = "CLOSED" self.failures = 0 return res except Exception as e: self.failures += 1 self.last_failure_time = now if self.failures >= self.threshold: self.state = "OPEN" raise e cb = CircuitBreaker(threshold=2, cooldown=0.05) def mock_service(should_fail=True): if should_fail: raise ConnectionError("Timeout") return "200_OK" # Trip the breaker for _ in range(2): try: cb.call(mock_service, True) except ConnectionError: pass assert cb.state == "OPEN" # Fast fail try: cb.call(mock_service, True) except RuntimeError as err: assert str(err) == "OPEN" # Probe recovery time.sleep(0.06) res = cb.call(mock_service, False) assert res == "200_OK" assert cb.state == "CLOSED" print(f"Circuit Breaker verified: state restored to {cb.state}") ``` </details>