Back
+30 XP
Concept

Module 32 Concurrency Mastery Capstone

Real-world asynchronous engines unify structured concurrency (`TaskGroup`), backpressured bounded channels (`Queue`), and context isolation (`contextvars`) into an ironclad pipeline that shuts down cleanly without leaking memory or tasks 🛡️🚀.

python
import asyncio
import contextvars

tenant_ctx = contextvars.ContextVar("tenant", default="DEFAULT")

async def producer(queue, batches):
    for batch in batches:
        await queue.put(batch)
    await queue.put(None) # Sentinel

async def consumer(queue, output):
    tenant = tenant_ctx.get()
    while True:
        item = await queue.get()
        if item is None:
            queue.task_done()
            break
        output.append(f"{tenant}_{item}")
        queue.task_done()

async def main():
    tenant_ctx.set("PROD_TIER")
    q = asyncio.Queue(maxsize=2)
    results = []

    async with asyncio.TaskGroup() as tg:
        tg.create_task(producer(q, [10, 20]))
        tg.create_task(consumer(q, results))

    print(f"Aggregated: {results} 🏁")

asyncio.run(main())

# 30. Module 32 Concurrency Mastery Capstone Amateur async code relies on naked `create_task()` fire-and-forget loops, leading to memory leaks, silent task drops, and corrupted shared mutable state under stress 🛑. ```python import asyncio import contextvars # Production pipeline: TaskGroup + Queue + ContextVar ``` ### The Production Pipeline Triad 1. **Structured Lifetimes (`TaskGroup`)**: Enforces lexical boundaries—if any step crashes, sibling workers are cleanly unwound without leaving zombie tasks roaming in memory 🛡️. 2. **Backpressured Queues (`asyncio.Queue(maxsize=N)`)**: Restricts in-flight memory consumption, ensuring consumers are never overwhelmed by high-rate producers. 3. **Contextual Isolation (`contextvars`)**: Preserves distributed trace IDs, multi-tenant keys, and auth contexts across async scheduling boundaries without risking cross-request contamination 🧬. --- ### Interactive Pop-Quiz 🧠 #### What design pattern guarantees that all background consumer tasks terminate cleanly when an async producer finishes its queue workload? - [ ] Enqueueing sentinel termination tokens (`None`) into the queue and awaiting context block exit 🛡️ - [ ] Terminating the Python process immediately using os._exit(0) - [ ] Sleeping for a fixed arbitrary duration and hoping all workers finished - [ ] Disconnecting the host network card to trigger timeout exceptions --- ### Practical Challenge 🎯 Construct an asynchronous pipeline with `tenant_var` set to `'TENANT_X'`. Create a bounded queue (`maxsize=2`). Spawn a producer that enqueues `['pkg_1', 'pkg_2']` followed by two `None` sentinels. Spawn two consumer workers inside `asyncio.TaskGroup()` that prefix each consumed item with the current tenant and append to a list `processed`. Print `sorted(processed)`. <details> <summary>Solution</summary> ```python import asyncio import contextvars tenant_var = contextvars.ContextVar("tenant", default="UNKNOWN") async def producer(q, items): for item in items: await q.put(item) await q.put(None) await q.put(None) async def consumer(q, processed): tid = tenant_var.get() while True: item = await q.get() if item is None: q.task_done() break processed.append(f"{tid}:{item}") q.task_done() async def main(): tenant_var.set("TENANT_X") q = asyncio.Queue(maxsize=2) processed = [] async with asyncio.TaskGroup() as tg: tg.create_task(producer(q, ["pkg_1", "pkg_2"])) tg.create_task(consumer(q, processed)) tg.create_task(consumer(q, processed)) print(sorted(processed)) asyncio.run(main()) ``` </details>

Checkpoint

What design pattern guarantees that all background consumer tasks terminate cleanly when an async producer finishes its queue workload? 🧠

Practice

Construct an asynchronous pipeline with `tenant_var` set to `'TENANT_X'`. Create a bounded queue (`maxsize=2`). Spawn a producer that enqueues `['pkg_1', 'pkg_2']` followed by two `None` sentinels. Spawn two consumer workers inside `asyncio.TaskGroup()` that prefix each consumed item with the current tenant and append to a list `processed`. Print `sorted(processed)`. 🎯

Terminal
01
Tap code blocks below to assemble
Available Blocks