Thread Pool Graceful Drain & Shutdown Handler
Killing a microservice with SIGTERM while background thread pools are actively processing orders causes dropped jobs and half-written records. A graceful drain handler closes the ingestion door to new submissions, lets existing tasks complete, and safely shuts down.
import threading
from concurrent.futures import ThreadPoolExecutor
class GracefulPool:
def __init__(self, workers=4):
self.pool = ThreadPoolExecutor(workers)
self.stopping = False
# Thread Pool Graceful Drain & Shutdown Handler When Kubernetes rolls out a new deployment, it sends a `SIGTERM` to the old pod. If your service terminates immediately, background worker threads that were in the middle of charging credit cards or writing database rows get severed midway through execution. 🛑🔌 A **Graceful Drain Handler** coordinates a clean exit: 1. Immediately flip an ingestion flag to reject any new incoming jobs with an informative exception. 2. Allow all currently running and enqueued tasks to complete their work. 3. Terminate worker threads cleanly and release all system resources. ```python from concurrent.futures import ThreadPoolExecutor import threading class GracefulPool: def __init__(self, workers: int = 4): self.executor = ThreadPoolExecutor(max_workers=workers) self.is_stopping = False self.lock = threading.Lock() ``` ## Pop Quiz Why must a thread pool manager reject new submissions before waiting for existing tasks during shutdown? - [ ] To bound the pending work and guarantee that the shutdown procedure will eventually terminate. - [ ] Because Python threads automatically reset their thread-local storage on SIGTERM. - [ ] To force the operating system to flush dirty CPU caches directly to disk. - [ ] Because ThreadPoolExecutor crashes if submitted tasks exceed available worker threads. ## Real-World Challenge Build a `GracefulThreadPool` wrapping `concurrent.futures.ThreadPoolExecutor`. Expose: - `submit(fn, *args, **kwargs)`: Enqueues a task, but raises `RuntimeError` once shutdown has begun. - `drain_and_shutdown() -> list`: Rejects future tasks, waits for existing scheduled futures to conclude, and returns a list of results from all successfully completed futures. <details> <summary>Solution</summary> ```python from concurrent.futures import ThreadPoolExecutor import threading import time class GracefulThreadPool: def __init__(self, max_workers: int = 4): self.executor = ThreadPoolExecutor(max_workers=max_workers) self.is_shutting_down = False self.lock = threading.Lock() self.tracked_futures = [] def submit(self, fn, *args, **kwargs): with self.lock: if self.is_shutting_down: raise RuntimeError("Pool is shutting down; new tasks rejected") future = self.executor.submit(fn, *args, **kwargs) self.tracked_futures.append(future) return future def drain_and_shutdown(self): with self.lock: self.is_shutting_down = True self.executor.shutdown(wait=True, cancel_futures=False) return [f.result() for f in self.tracked_futures if not f.cancelled() and not f.exception()] # Verification def work(val: int): time.sleep(0.01) return val * 10 pool = GracefulThreadPool(max_workers=2) f1 = pool.submit(work, 1) f2 = pool.submit(work, 2) results = pool.drain_and_shutdown() assert sorted(results) == [10, 20] rejected = False try: pool.submit(work, 3) except RuntimeError: rejected = True assert rejected is True ``` </details>