Fix Subprocess Zombie Processes via os.waitpid
When a child subprocess terminates, it remains in the OS process table as a defunct zombie until the parent process reads its exit status. Neglecting proc.wait() or os.waitpid in long-lived background daemons exhausts process IDs, blocking subsequent process creation.
import subprocess
proc = subprocess.Popen(['echo', 'ping'], stdout=subprocess.PIPE)
# communicate() internally waits and reaps the child process exit status
out, err = proc.communicate()
# Fix Subprocess Zombie Processes via os.waitpid When your microservice launches background FFmpeg encoders or shell tasks via `subprocess.Popen(cmd)` without calling `.wait()` or `.communicate()`, the child process eventually finishes and dies. But it doesn't vanish from the operating system. 🧟♂️💀 It transforms into a **Zombie Process** (marked `<defunct>` in `ps aux`). The Linux kernel holds its exit status in the system process table waiting for the parent to inspect it. If your service spawns thousands of un-reaped subprocesses, it consumes all available system Process IDs (PIDs), causing subsequent `fork()` calls to crash with `OSError: [Errno 12] Cannot allocate memory`. ```python import subprocess # The Antidote: communicate() reads buffers and calls wait() internally proc = subprocess.Popen(["ls", "-la"], stdout=subprocess.PIPE, stderr=subprocess.PIPE) stdout, stderr = proc.communicate(timeout=5.0) exit_code = proc.returncode ``` ## Pop Quiz Why do zombie (`<defunct>`) processes accumulate if a parent process never waits on its child subprocesses? - [ ] The OS kernel retains the child process descriptor and exit code until the parent acknowledges it. - [ ] Python delays subprocess cleanup until all CPU hardware threads become idle. - [ ] The child process memory is transferred to the system init daemon automatically. - [ ] Zombie processes remain alive to handle re-execution requests from standard input. ## Real-World Challenge Implement `safe_spawn_and_reap(cmd: list, timeout_sec: float = 2.0) -> dict`. It must execute the command, capture stdout and stderr, guarantee that the process is waited upon (preventing zombie state even if timing out), and return `{"returncode": int, "stdout": str, "stderr": str}`. <details> <summary>Solution</summary> ```python import subprocess def safe_spawn_and_reap(cmd: list, timeout_sec: float = 2.0) -> dict: proc = subprocess.Popen( cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True ) try: stdout_data, stderr_data = proc.communicate(timeout=timeout_sec) return { "returncode": proc.returncode, "stdout": stdout_data.strip(), "stderr": stderr_data.strip() } except subprocess.TimeoutExpired: # Kill the hung process and reap it so it does not linger as a zombie proc.kill() stdout_data, stderr_data = proc.communicate() return { "returncode": -1, "stdout": stdout_data.strip(), "stderr": "Execution timed out" } # Verification res = safe_spawn_and_reap(["echo", "reaped_cleanly"]) assert res["returncode"] == 0 assert res["stdout"] == "reaped_cleanly" assert res["stderr"] == "" ``` </details>