Back
+25 XP
Concept

Health Check Architecture: Liveness vs Readiness vs Startup Probes

Container platforms like Kubernetes rely on three discrete probes: Startup Probes allow slow initialization without premature termination, Readiness Probes control whether load balancers route ingress traffic to the instance, and Liveness Probes detect unrecoverable internal deadlocks to trigger container restarts 💓🛡️🚀.

python
class ServiceProbes:
    def __init__(self):
        self.initialized = False
        self.db_connected = False
        self.healthy = True

    def liveness(self):
        # Process is alive; returning 500 triggers container restart
        return (200, "OK") if self.healthy else (500, "DEADLOCKED")

    def readiness(self):
        # Ready for traffic; 503 removes pod from ingress routing
        if self.initialized and self.db_connected:
            return (200, "READY")
        return (503, "NOT_READY")

probes = ServiceProbes()
print("Initial Readiness:", probes.readiness())
probes.initialized = True
probes.db_connected = True
print("Active Readiness:", probes.readiness())

# 95. Health Check Architecture: Liveness vs Readiness vs Startup Probes Lumping all health checks into a single `/health` endpoint that returns 200 OK is one of the most destructive mistakes in distributed systems engineering. Conflating liveness with readiness leads to cascading restart storms 🛑. ```python # The Probe Hierarchy: # 1. Startup: "Am I done loading models, warm caches, or parsing schemas?" # Failure -> Give it more time (or kill if initial deadline exceeded). # 2. Readiness: "Can I take user traffic right now?" # Failure -> Remove pod IP from Load Balancer (do NOT kill process). # 3. Liveness: "Is my internal event loop deadlocked or corrupted?" # Failure -> Kill container and spin up a fresh replica. ``` ### The Cascading Restart Anti-Pattern If your database goes down and your `liveness` probe queries `SELECT 1;`, your database failure immediately triggers Kubernetes to kill every single Python application pod at once! When the database recovers, all pods attempt cold-start reconnections simultaneously, immediately knocking the database down again in a thundering herd death spiral. - **Rule**: Liveness must ONLY verify local process health (e.g. event loop ticks, free memory). - **Rule**: Readiness verifies downstream connectivity before taking traffic 🛡️⚡. --- ### Interactive Pop-Quiz 🧠 #### What catastrophe occurs if a service returns 500 from its Liveness probe when its downstream database is temporarily unavailable? - [ ] A cascading restart storm: all container pods fail liveness and restart simultaneously, compounding database overload 💀 - [ ] The database automatically recovers in under 1 millisecond - [ ] The Linux kernel clears all disk partitions - [ ] The orchestrator converts the container into a static HTML file --- ### Practical Challenge 🎯 Implement `HealthChecker` with attributes `started`, `ready`, `alive`. Provide methods `check_startup() -> (int, str)`, `check_readiness() -> (int, str)`, and `check_liveness() -> (int, str)` returning 200 on success and 503/500 on failure (`STARTING_UP`, `NOT_READY`, `DEADLOCKED`). Verify lifecycle transitions from unstarted to ready, and death on `alive=False`. Print the results. <details> <summary>Solution</summary> ```python class HealthChecker: def __init__(self): self.started = False self.ready = False self.alive = True def check_startup(self) -> tuple[int, str]: if self.started: return 200, "STARTUP_OK" return 503, "STARTING_UP" def check_readiness(self) -> tuple[int, str]: if self.started and self.ready: return 200, "READY" return 503, "NOT_READY" def check_liveness(self) -> tuple[int, str]: if self.alive: return 200, "ALIVE" return 500, "DEADLOCKED" hc = HealthChecker() assert hc.check_startup() == (503, "STARTING_UP") assert hc.check_readiness() == (503, "NOT_READY") hc.started = True assert hc.check_startup() == (200, "STARTUP_OK") assert hc.check_readiness() == (503, "NOT_READY") hc.ready = True assert hc.check_readiness() == (200, "READY") assert hc.check_liveness() == (200, "ALIVE") hc.alive = False assert hc.check_liveness() == (500, "DEADLOCKED") print("HealthChecker verified: 200 and 503/500 responses operational") ``` </details>

Checkpoint

What catastrophe occurs if a service returns 500 from its Liveness probe when its downstream database is temporarily unavailable? 🧠

Practice

Implement `HealthChecker` with attributes `started`, `ready`, `alive`. Provide methods `check_startup() -> (int, str)`, `check_readiness() -> (int, str)`, and `check_liveness() -> (int, str)` returning 200 on success and 503/500 on failure (`STARTING_UP`, `NOT_READY`, `DEADLOCKED`). Verify lifecycle transitions from unstarted to ready, and death on `alive=False`. Print the results. 🎯

Terminal
01
Tap code blocks below to assemble
Available Blocks