Back
+25 XP
Concept

Graceful Degradation & Shedding Load under Extreme Congestion

Attempting to serve every request during an extreme traffic surge causes total system collapse: latency explodes, queues overflow, and throughput drops to zero. Load shedding actively rejects low-priority traffic early, preserving critical core transactions like payments and health probes 🛡️🌊⚡.

python
class PriorityShedder:
    def __init__(self, limit=1000):
        self.limit = limit
        self.inflight = 0

    def admit(self, priority):
        # Load > 80% sheds low-priority traffic; 100% drops all
        usage = self.inflight / self.limit
        if usage >= 1.0: return False
        if usage >= 0.8 and priority == "LOW": return False
        self.inflight += 1
        return True

shedder = PriorityShedder(limit=10)
shedder.inflight = 9
print("Admit LOW:", shedder.admit("LOW"))
print("Admit CRITICAL:", shedder.admit("CRITICAL"))

# 103. Graceful Degradation & Shedding Load under Extreme Congestion When a viral event or marketing campaign unleashes 10x normal traffic, a naive server tries to accept and process every single incoming TCP socket. As thread pools starve and memory fills with queued request buffers, garbage collection pauses spike, timeouts trigger everywhere, and the service collapses to 0 requests/sec 🛑. ```python # The Law of Congestion Collapse: # Goodput (useful completed work) drops off a cliff as concurrency exceeds peak capacity: # Throughput ^ # | /---\ (Peak Capacity) # | / \ # | / \ (Congestion Collapse -> 0 ops/sec) # +-----+---------+---------------------> Ingress Load ``` ### Strategic Load Shedding Principles 1. **Reject Early and Cheaply**: When internal queues or concurrency approach capacity, reject excess requests at the edge in microseconds. Never let a request sit in a 10-second queue only to time out when the user has already refreshed the page. 2. **Priority-Tiered Admission**: Categorize traffic into tiers: - **Critical (Tier 1)**: User checkout, payment authorization, health probes. - **Standard (Tier 2)**: Search queries, user feed browsing. - **Low (Tier 3)**: Analytics beacons, background synchronizations, recommendations. 3. **Graceful Degradation Fallbacks**: If the ML recommendation engine is overloaded, return static curated top-10 items instead of throwing a 500 error 🛡️⚡. --- ### Interactive Pop-Quiz 🧠 #### Why is rejecting excess requests immediately with HTTP 503/429 better than letting them queue up in memory? - [ ] Queueing consumes RAM and causes tail-latency timeouts for all users, whereas early shedding preserves system stability and low latency for admitted requests 🛡️ - [ ] Queued requests corrupt the CPU floating-point registers - [ ] HTTP 503 causes clients to permanently disconnect from the internet - [ ] Queueing requires dedicated GPU acceleration --- ### Practical Challenge 🎯 Implement `LoadShedder` with `max_concurrency: int` and `shed_threshold: float = 0.8`. Accept integer priority (1=LOW, 2=MED, 3=HIGH, 4=CRITICAL). Provide `admit(priority: int) -> bool` that rejects everything at $\ge 1.0$ load factor, rejects priority $< 3$ at $\ge shed\_threshold$ load factor, and admits all otherwise. Verify with capacity 10 and inflight count 8 that LOW is dropped and CRITICAL is admitted. Print results. <details> <summary>Solution</summary> ```python class LoadShedder: def __init__(self, max_concurrency: int, shed_threshold: float = 0.8): self.max_concurrency = max_concurrency self.shed_threshold = shed_threshold self.active_requests = 0 def admit(self, priority: int) -> bool: load_factor = self.active_requests / self.max_concurrency if load_factor >= 1.0: return False if load_factor >= self.shed_threshold: # Under heavy load: only admit HIGH (3) and CRITICAL (4) return priority >= 3 return True shedder = LoadShedder(max_concurrency=10, shed_threshold=0.8) shedder.active_requests = 8 # 80% capacity assert shedder.admit(1) is False # LOW rejected assert shedder.admit(2) is False # MED rejected assert shedder.admit(3) is True # HIGH admitted assert shedder.admit(4) is True # CRITICAL admitted shedder.active_requests = 10 # 100% capacity assert shedder.admit(4) is False # Everything shed print("LoadShedder verified: graceful degradation under congestion operational") ``` </details>

Checkpoint

Why is rejecting excess requests immediately with HTTP 503/429 better than letting them queue up in memory? 🧠

Practice

Implement `LoadShedder` with `max_concurrency: int` and `shed_threshold: float = 0.8`. Accept integer priority (1=LOW, 2=MED, 3=HIGH, 4=CRITICAL). Provide `admit(priority: int) -> bool` that rejects everything at >= 1.0 load factor, rejects priority < 3 at >= shed_threshold load factor, and admits all otherwise. Verify with capacity 10 and inflight count 8 that LOW is dropped and CRITICAL is admitted. Print results. 🎯

Terminal
01
Tap code blocks below to assemble
Available Blocks