Back
+30 XP
Concept

Non-Blocking File Watcher via Async Generators

Tailing growing log files or streaming network sockets with standard synchronous loops freezes the event loop. An asynchronous generator uses non-blocking polling and sleep yields to stream new lines in real time without blocking concurrent tasks.

python
import asyncio

async def stream_log_lines(source, poll_sec=0.1):
    while True:
        line = source.read_line()
        if line:
            yield line
        else:
            await asyncio.sleep(poll_sec)

# Non-Blocking File Watcher via Async Generators Writing `while True: line = f.readline()` on a server tailing audit logs blocks the entire thread whenever no new lines exist. In an `asyncio` server, blocking synchronous file reads starves incoming HTTP and WebSocket connections. 🛑🌊 By encapsulating stream polling inside an **asynchronous generator** (`async def ... yield`), your function yields new data immediately when present, and safely relinquishes event loop execution with `await asyncio.sleep()` when idle. ```python import asyncio async def async_line_tail(source, poll_interval: float = 0.05): while True: items = source.poll_new() for item in items: yield item await asyncio.sleep(poll_interval) ``` ## Pop Quiz What is the primary benefit of wrapping an I/O polling loop in an async generator (`async for`)? - [ ] It yields control back to the event loop on idle ticks, keeping the application fully responsive. - [ ] It compresses log payloads automatically using gzip before storing them in memory. - [ ] It prevents OS file system notifications from raising inotify queue overflow errors. - [ ] It disables disk cache flushing to boost write throughput by 10x. ## Real-World Challenge Build `async_file_watcher(source, poll_interval: float)`. The `source` object provides `read_new()` which returns a list of newly buffered strings. Your generator must yield each string sequentially and pause with `await asyncio.sleep(poll_interval)` when no new strings are available. <details> <summary>Solution</summary> ```python import asyncio class MockLogSource: def __init__(self): self.buffer = [] def append(self, text: str): self.buffer.append(text) def read_new(self): new_items = list(self.buffer) self.buffer.clear() return new_items async def async_file_watcher(source, poll_interval: float = 0.01): while True: lines = source.read_new() for line in lines: yield line await asyncio.sleep(poll_interval) # Verification async def run_test(): source = MockLogSource() collected = [] async def consumer(): async for entry in async_file_watcher(source, poll_interval=0.01): collected.append(entry) if len(collected) == 3: break task = asyncio.create_task(consumer()) await asyncio.sleep(0.01) source.append("SYS_BOOT") source.append("CONFIG_LOADED") await asyncio.sleep(0.02) source.append("READY_ACCEPT") await task assert collected == ["SYS_BOOT", "CONFIG_LOADED", "READY_ACCEPT"] asyncio.run(run_test()) ``` </details>