Concurrent Futures Timeout Watchdog
Offloading blocking legacy code to ThreadPoolExecutor without strict timeouts will eventually lock every thread in the pool when remote sockets hang. A watchdog inspects future.result(timeout=...) to report hung threads and cancel unstarted work.
from concurrent.futures import ThreadPoolExecutor, TimeoutError
with ThreadPoolExecutor(max_workers=2) as pool:
fut = pool.submit(legacy_io)
try:
res = fut.result(timeout=1.0)
except TimeoutError:
fut.cancel()
# Concurrent Futures Timeout Watchdog If you offload blocking synchronous I/O or legacy SDK calls to a `ThreadPoolExecutor` without timeouts, a single hung socket will freeze that worker thread forever. Given enough time, all workers freeze and your entire backend hangs. ⏱️💀 Because standard Python threads cannot be forcibly killed while executing native C extensions or blocking syscalls, a **Watchdog pattern** tracks completion times, flags stalled operations, and cancels any queued futures before they waste further compute resources. ```python from concurrent.futures import ThreadPoolExecutor, TimeoutError with ThreadPoolExecutor(max_workers=4) as pool: future = pool.submit(slow_blocking_task) try: data = future.result(timeout=2.0) except TimeoutError: future.cancel() ``` ## Pop Quiz Why does `future.cancel()` fail to terminate a thread that is already running blocking I/O? - [ ] In Python, threads running blocking C syscalls or unyielding code cannot be preemptively killed from userspace. - [ ] The thread pool manager converts cancelled futures into high-priority daemon subprocesses. - [ ] The GIL locks all CPU registers until the blocking socket returns an EOF marker. - [ ] Cancelling an active future raises an unrecoverable SIGSEGV in glibc. ## Real-World Challenge Write `run_with_watchdog(callables: list, timeout_sec: float)`. It must submit all callables to a `ThreadPoolExecutor(max_workers=4)`. For each task, record whether it completed successfully, timed out, or threw an exception, returning a list of `{"status": ..., "value": ...}` dictionaries. <details> <summary>Solution</summary> ```python from concurrent.futures import ThreadPoolExecutor, TimeoutError import time def run_with_watchdog(callables, timeout_sec: float): results = [] with ThreadPoolExecutor(max_workers=4) as executor: futures = [executor.submit(fn) for fn in callables] for f in futures: try: val = f.result(timeout=timeout_sec) results.append({"status": "completed", "value": val}) except TimeoutError: f.cancel() results.append({"status": "timed_out", "value": None}) except Exception as e: results.append({"status": "error", "value": str(e)}) return results # Verification def quick_task(): time.sleep(0.01) return 42 def hanging_task(): time.sleep(0.2) return "done" reports = run_with_watchdog([quick_task, hanging_task], timeout_sec=0.05) assert reports[0] == {"status": "completed", "value": 42} assert reports[1] == {"status": "timed_out", "value": None} ``` </details>