Tarball Header Block Parser (POSIX ustar)
The Unix tar archive format organizes files into contiguous 512-byte blocks without central directory records. Each file begins with a 512-byte POSIX ustar header containing null-terminated file names and base-8 octal ASCII strings for sizes, modes, and timestamps.
# Extract file size from 12-byte octal ASCII slice:
size_octal = block[124:136].strip(b'\x00 ')
file_size = int(size_octal, 8) if size_octal else 0
# Tarball Header Block Parser (POSIX ustar) Before zip files had central directories, Unix administrators backed up magnetic tapes using `tar` (Tape Archive). The format was designed for sequential streaming: every file consists of a **512-byte header block**, followed by file contents padded to 512-byte boundaries. 📦💾 The POSIX **ustar** standard defines the header layout: - `0..99`: File name (ASCII, null-terminated) - `100..107`: File mode (octal string) - `108..115`: Owner UID (octal string) - `116..123`: Owner GID (octal string) - `124..135`: File size in bytes (12-byte octal ASCII string) - `136..147`: Modification time (octal timestamp) - `156`: Type flag (`'0'` or `\0` for regular file, `'5'` for directory) - `257..262`: Magic string (`"ustar\0"` or `"ustar "`) ```python # Decoding octal strings: size_bytes = header[124:136].strip(b"\x00 ") size = int(size_bytes, 8) if size_bytes else 0 ``` ## Pop Quiz How does the POSIX ustar tarball format encode numerical metadata such as file sizes and file permissions? - [ ] As null-padded or space-terminated ASCII character strings representing base-8 octal numbers. - [ ] As 64-bit IEEE floating-point numbers encoded in little-endian binary. - [ ] As variable-length UTF-8 strings delimited by colon characters. - [ ] As zlib-compressed binary bitmasks stored in the archive trailer. ## Real-World Challenge Implement `parse_tar_header(block: bytes) -> dict | None`. - If the block is 512 null bytes (`b"\x00" * 512`), return `None` (end of archive marker). - Otherwise, extract `filename` (str), `size` (int), `type` (`"dir"` if typeflag is `'5'` else `"file"`), and `magic` (str). Validate that `len(block) == 512`. <details> <summary>Solution</summary> ```python def parse_tar_header(block: bytes) -> dict | None: if len(block) != 512: raise ValueError(f"Expected 512-byte TAR block, got {len(block)}") if block == b"\x00" * 512: return None filename = block[0:100].split(b"\x00")[0].decode("utf-8", errors="replace") raw_size = block[124:136].strip(b"\x00 ") size = int(raw_size, 8) if raw_size else 0 typeflag = chr(block[156]) entry_type = "dir" if typeflag == "5" else "file" magic = block[257:263].split(b"\x00")[0].decode("ascii", errors="replace") return { "filename": filename, "size": size, "type": entry_type, "magic": magic } # Verification # Craft synthetic 512-byte block block = bytearray(512) block[0:16] = b"usr/bin/python3\x00" block[124:135] = b"00000000200" # 200 in octal = 128 in decimal block[156:157] = b"0" # Regular file block[257:262] = b"ustar" parsed = parse_tar_header(bytes(block)) assert parsed["filename"] == "usr/bin/python3" assert parsed["size"] == 128 assert parsed["type"] == "file" assert parsed["magic"] == "ustar" # Verify EOF block assert parse_tar_header(b"\x00" * 512) is None ``` </details>