Shared Memory with multiprocessing.shared_memory
Pipes and queues serialize data, incurring heavy pickling CPU overhead. `multiprocessing.shared_memory.SharedMemory` maps raw zero-copy POSIX memory blocks directly across isolated process address spaces 🏎️.
from multiprocessing import shared_memory
shm = shared_memory.SharedMemory(create=True, size=16)
try:
shm.buf[:4] = b"FAST"
# Attach reader to same memory segment:
reader = shared_memory.SharedMemory(name=shm.name)
print(f"Read: {bytes(reader.buf[:4]).decode()} 🏎️")
reader.close()
finally:
shm.close()
shm.unlink() # Deallocate from OS /dev/shm
# 78. Shared Memory with multiprocessing.shared_memory Pipes and queues serialize data, incurring heavy pickling CPU overhead. `multiprocessing.shared_memory.SharedMemory` maps raw zero-copy POSIX memory blocks directly across isolated process address spaces 🏎️. ```python from multiprocessing import shared_memory shm = shared_memory.SharedMemory(create=True, size=16) try: shm.buf[:4] = b"FAST" # Attach reader to same memory segment: reader = shared_memory.SharedMemory(name=shm.name) print(f"Read: {bytes(reader.buf[:4]).decode()} 🏎️") reader.close() finally: shm.close() shm.unlink() # Deallocate from OS /dev/shm ``` High-performance data science and computer vision pipelines wrap `shm.buf` inside NumPy arrays (`np.ndarray(..., buffer=shm.buf)`) to process massive 4K video frames without copying a single byte between processes 🚀! --- ### Question Why must you explicitly call `shm.unlink()` on a POSIX shared memory segment? 🧠 - [ ] a. `close()` only detaches the file descriptor; `unlink()` tells the OS kernel to destroy the RAM segment 🏎️ - [ ] b. It rewrites all stored memory bytes to null bytes for security compliance - [ ] c. It terminates all currently attached child processes immediately - [ ] d. Calling unlink converts raw memory blocks into disk-backed swap files --- ### Challenge Allocate a `SharedMemory(create=True, size=8)`. Write `b'PYTHON3'` into its buffer. Read it back, close and unlink the segment. Print the decoded string. 🎯 <details> <summary>Solution</summary> ```python from multiprocessing import shared_memory shm = shared_memory.SharedMemory(create=True, size=8) try: shm.buf[:7] = b'PYTHON3' res = bytes(shm.buf[:7]).decode() print(res) finally: shm.close() shm.unlink() ``` </details>