Back
+30 XP
Concept

Atomic File Writer with Tempfile Swap & fsync

Writing directly to critical configuration files risks catastrophic corruption if a power loss or crash occurs mid-write. An atomic file write streams to a temporary file on the same filesystem, flushes disk buffers with os.fsync, and executes an atomic POSIX os.replace.

python
import os, tempfile

temp_fd, temp_path = tempfile.mkstemp(dir=target_dir)
with open(temp_fd, 'w') as f:
    f.write(data)
    f.flush()
    os.fsync(f.fileno())
os.replace(temp_path, target_path)

# Atomic File Writer with Tempfile Swap & fsync Never write directly to critical configuration or state files with `open(path, "w")`. If your server loses power, crashes, or gets killed with `SIGKILL` halfway through writing `database.json`, the file will contain truncated, half-written garbage. 🛑💾 The **Atomic Write Pattern** prevents this: 1. Create a temporary file in the **same directory** as the target file. 2. Write data and invoke `f.flush()`. 3. Call `os.fsync(f.fileno())` to force the OS kernel and drive hardware to flush dirty write caches to physical media. 4. Call `os.replace(temp_path, target_path)`: on POSIX systems, `rename(2)` is an atomic directory metadata swap. Either the old file remains or the complete new file appears—no half-written states ever exist. ```python import os import tempfile dir_name = os.path.dirname(os.path.abspath(filepath)) fd, tmp_path = tempfile.mkstemp(dir=dir_name, prefix=".tmp_") with open(fd, "w", encoding="utf-8") as f: f.write(content) f.flush() os.fsync(f.fileno()) os.replace(tmp_path, filepath) ``` ## Pop Quiz Why must the temporary file reside on the exact same filesystem directory as the final target file? - [ ] The POSIX rename/replace syscall is only atomic when moving files within the same filesystem partition. - [ ] Cross-device file moves automatically delete the operating system inode table. - [ ] To prevent Python from converting text encoding from UTF-8 to Latin-1. - [ ] Because Linux file permissions cannot be copied across separate hard drive platters. ## Real-World Challenge Implement `atomic_write(filepath: str, data: str)`. - Use `tempfile.mkstemp` in the target file's parent directory. - Ensure `os.fsync` flushes disk buffers. - Atomically replace the target file via `os.replace`. - Clean up the temporary file if an exception occurs during writing. <details> <summary>Solution</summary> ```python import os import tempfile def atomic_write(filepath: str, data: str): abs_path = os.path.abspath(filepath) dir_name = os.path.dirname(abs_path) or "." os.makedirs(dir_name, exist_ok=True) temp_fd, temp_path = tempfile.mkstemp(dir=dir_name, prefix=".tmp_atomic_") try: with open(temp_fd, "w", encoding="utf-8") as f: f.write(data) f.flush() os.fsync(f.fileno()) os.replace(temp_path, abs_path) except Exception: if os.path.exists(temp_path): try: os.remove(temp_path) except OSError: pass raise # Verification target_test = "/tmp/test_atomic_production.conf" atomic_write(target_test, "CONFIG_V1=ENABLED") with open(target_test, "r") as f: assert f.read() == "CONFIG_V1=ENABLED" # Overwrite atomically atomic_write(target_test, "CONFIG_V2=ACTIVE") with open(target_test, "r") as f: assert f.read() == "CONFIG_V2=ACTIVE" os.remove(target_test) ``` </details>