Back
+30 XP
Concept

File Lock Guard via fcntl.flock

When multiple independent OS processes write to the same file or sqlite database, threading locks cannot coordinate them because memory is isolated. The fcntl.flock advisory lock system locks files at the operating system kernel level, supporting shared read and exclusive write locks.

python
import fcntl, os

fd = os.open('app.lock', os.O_CREAT | os.O_RDWR)
# Exclusive non-blocking lock:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)

# File Lock Guard via fcntl.flock If you have 4 Gunicorn or Uvicorn worker processes running on the same Linux host and all 4 try to write to `shared_state.json` simultaneously, using `threading.Lock()` does nothing. Each process lives in its own completely isolated virtual memory sandbox. 🛑🔒 To synchronize multiple independent OS processes, you must ask the **operating system kernel** to lock the file table entry via `fcntl.flock`: - `fcntl.LOCK_EX`: Exclusive write lock (only 1 process allowed). - `fcntl.LOCK_SH`: Shared read lock (multiple readers allowed). - `fcntl.LOCK_NB`: Non-blocking (raises `BlockingIOError` or `OSError` immediately if held by another process). - `fcntl.LOCK_UN`: Release lock. ```python import fcntl import os fd = os.open("app.lock", os.O_CREAT | os.O_RDWR) fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB) ``` ## Pop Quiz Why cannot a `threading.Lock` synchronize file writes between separate Python processes? - [ ] Operating system processes have isolated virtual memory address spaces; threading locks only exist in the memory of a single process. - [ ] Threading locks automatically delete files when acquired across fork boundaries. - [ ] The Linux kernel only permits inter-process synchronization via raw TCP sockets. - [ ] Because Python threading locks expire after 50 milliseconds of idle CPU time. ## Real-World Challenge Implement `@contextmanager file_flock_guard(filepath: str, exclusive: bool = True, non_blocking: bool = False)`. - Open the file using low-level `os.open`. - Apply `fcntl.flock` with appropriate flags. - In `finally`, unlock and close the file descriptor even if an error is raised inside the `with` block. <details> <summary>Solution</summary> ```python import fcntl import os from contextlib import contextmanager @contextmanager def file_flock_guard(filepath: str, exclusive: bool = True, non_blocking: bool = False): flags = fcntl.LOCK_EX if exclusive else fcntl.LOCK_SH if non_blocking: flags |= fcntl.LOCK_NB fd = os.open(filepath, os.O_CREAT | os.O_RDWR) try: fcntl.flock(fd, flags) yield fd finally: try: fcntl.flock(fd, fcntl.LOCK_UN) except OSError: pass os.close(fd) # Verification test_lock = "/tmp/test_kernel_flock.lock" # 1. Exclusive acquisition with file_flock_guard(test_lock, exclusive=True) as fd1: # 2. Conflicting non-blocking acquisition must raise BlockingIOError or OSError contention_blocked = False try: with file_flock_guard(test_lock, exclusive=True, non_blocking=True) as fd2: pass except (BlockingIOError, OSError): contention_blocked = True assert contention_blocked is True, "Second lock acquisition should have been blocked!" # Clean up if os.path.exists(test_lock): os.remove(test_lock) ``` </details>