Back
+20 XP
Concept

Inter-Process Queues with multiprocessing.Queue

`multiprocessing.Queue` enables safe multi-producer, multi-consumer communication across process boundaries. It serializes objects using `pickle` and feeds them through underlying OS pipes using internal worker threads 📬.

python
import multiprocessing

def producer(q):
    for job in ["task_a", "task_b", "STOP"]:
        q.put(job)

if __name__ == "__main__":
    queue = multiprocessing.Queue()
    p = multiprocessing.Process(target=producer, args=(queue,))
    p.start()
    while (task := queue.get()) != "STOP":
        print(f"Consumed: {task} 📬")
    p.join()

# 77. Inter-Process Queues with multiprocessing.Queue `multiprocessing.Queue` enables safe multi-producer, multi-consumer communication across process boundaries. It serializes objects using `pickle` and feeds them through underlying OS pipes using internal worker threads 📬. ```python import multiprocessing def producer(q): for job in ["task_a", "task_b", "STOP"]: q.put(job) if __name__ == "__main__": queue = multiprocessing.Queue() p = multiprocessing.Process(target=producer, args=(queue,)) p.start() while (task := queue.get()) != "STOP": print(f"Consumed: {task} 📬") p.join() ``` If a process puts data into a queue and exits immediately before the background feeder thread flushes bytes to the pipe, the process can hang. Calling `queue.close()` and `queue.join_thread()` ensures clean exits ⚡! --- ### Question Why must any data placed into a `multiprocessing.Queue` be picklable? 🧠 - [ ] a. Objects must be serialized into bytes to travel across isolated OS process memory boundaries 📬 - [ ] b. Unpickled objects cause memory address bus deadlocks in the kernel - [ ] c. The multiprocessing module converts unpickled data into JSON strings - [ ] d. CPython forbids sending objects larger than 64 bytes between threads --- ### Challenge Use `multiprocessing.Queue` to send `10` and `20` from a worker process. In the parent process, retrieve both, sum them, join the process, and print the total (`30`). 🎯 <details> <summary>Solution</summary> ```python import multiprocessing def worker(q): q.put(10) q.put(20) if __name__ == '__main__': q = multiprocessing.Queue() p = multiprocessing.Process(target=worker, args=(q,)) p.start() total = q.get() + q.get() p.join() print(total) ``` </details>

Checkpoint

Why must any data placed into a `multiprocessing.Queue` be picklable? 🧠

Practice

Use `multiprocessing.Queue` to send `10` and `20` from a worker process. In the parent process, retrieve both, sum them, join the process, and print the total (`30`). 🎯

Terminal
01
Tap code blocks below to assemble
Available Blocks