Rate-Limited Async Worker Pool
Blasting 50,000 unmetered coroutines into asyncio.gather nukes your OS file descriptors or gets your IP banned. A rate-limited worker pool caps concurrency with an asyncio.Semaphore and enforces dispatch intervals with a high-resolution monotonic timer.
import asyncio
async def rate_limited_task(sem: asyncio.Semaphore, item: int):
async with sem:
await asyncio.sleep(0.01)
return item * 2
# Rate-Limited Async Worker Pool Spamming `asyncio.gather(*[heavy_io() for _ in range(50000)])` is the fastest way to crater your server with `OSError: [Errno 24] Too many open files` or get permanently rate-limited by remote APIs. Production async systems must cap concurrency and throttle request dispatch rates. 🛑⚡ An async worker pool balances throughput against system resource limits by combining an `asyncio.Semaphore` (capping active in-flight coroutines) with a monotonic dispatch timestamp (guaranteeing minimum delay between dispatches). ```python import asyncio async def bounded_worker(sem: asyncio.Semaphore, task_id: int): async with sem: await asyncio.sleep(0.05) return f"Task {task_id} completed" ``` ## Pop Quiz Why is an `asyncio.Semaphore` preferred over spawning infinite raw tasks in `asyncio.gather`? - [ ] It bounds in-flight coroutines, preventing socket starvation and OS file descriptor exhaustion. - [ ] It instructs the Python GIL to multi-thread coroutines across all CPU cores simultaneously. - [ ] It guarantees microsecond-level atomic memory synchronization between separate OS worker threads. - [ ] It auto-replaces failed network connections with cached synthetic HTTP 200 responses. ## Real-World Challenge Build an `AsyncPool` class that accepts `max_concurrency: int` and `rate_limit_per_sec: float`. Its `map(coro_fn, items)` method must execute coroutines across items respecting both constraints and return all results in original input order. <details> <summary>Solution</summary> ```python import asyncio class AsyncPool: def __init__(self, max_concurrency: int, rate_limit_per_sec: float): self.sem = asyncio.Semaphore(max_concurrency) self.min_interval = 1.0 / rate_limit_per_sec if rate_limit_per_sec > 0 else 0.0 self.last_dispatch = 0.0 self.lock = asyncio.Lock() async def _execute(self, coro_fn, item): async with self.sem: async with self.lock: loop = asyncio.get_running_loop() now = loop.time() elapsed = now - self.last_dispatch if elapsed < self.min_interval: await asyncio.sleep(self.min_interval - elapsed) self.last_dispatch = loop.time() return await coro_fn(item) async def map(self, coro_fn, items): tasks = [asyncio.create_task(self._execute(coro_fn, item)) for item in items] return await asyncio.gather(*tasks) # Verification async def run_test(): pool = AsyncPool(max_concurrency=2, rate_limit_per_sec=50.0) async def square(x): await asyncio.sleep(0.01) return x ** 2 res = await pool.map(square, [1, 2, 3, 4, 5]) assert res == [1, 4, 9, 16, 25], f"Unexpected result: {res}" asyncio.run(run_test()) ``` </details>