Back
+30 XP
Concept

Content-Addressable Storage (CAS) with SHA-256

Content-Addressable Storage (CAS) indexes and retrieves data blobs by their cryptographic hash rather than arbitrary file paths. This provides automatic deduplication, tamper-evidence, and idempotent writes, forming the foundational architecture of Git, IPFS, and Docker layer caches.

python
import hashlib

def get_hash(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()

# Content-Addressable Storage (CAS) with SHA-256 Traditional filesystems store data by human-defined paths: `/photos/vacation.jpg`. If you save the same 10 MB image in 5 different folders, the filesystem burns 50 MB of disk space. Even worse, if a bit flips on disk, you have no way to know the file corrupted. 🛡️📦 **Content-Addressable Storage (CAS)** flips the model: the content *is* the address. - When you store a byte array, compute its cryptographic hash: $\text{hash} = \text{SHA-256}(\text{data})$. - Store the data indexed under that hash. - If 1,000 users upload the exact same file, it is stored **once**. - Upon retrieval, recompute the hash. If the hashes don't match, silent bit rot or tampering is immediately detected. ```python import hashlib def cas_key(blob: bytes) -> str: return hashlib.sha256(blob).hexdigest() ``` ## Pop Quiz What primary operational benefit does Content-Addressable Storage (CAS) provide to distributed backup systems? - [ ] Identical data blocks naturally produce identical hash keys, enabling instant global deduplication and tamper verification. - [ ] It compresses data blocks using quantum error-correction algorithms. - [ ] It forces file storage systems to allocate contiguous physical disk sectors. - [ ] Because CAS files cannot be read without TLS 1.3 client certificates. ## Real-World Challenge Build a production-grade `ContentAddressableStore`: - `store(data: bytes) -> str`: Hashes the payload with SHA-256, stores it, and returns the hex digest. - `retrieve(content_hash: str) -> bytes`: Returns the stored payload. If the recomputed hash fails to match `content_hash`, raise `ValueError("Integrity check failed: corrupted blob")`. - `exists(content_hash: str) -> bool`: Checks existence. <details> <summary>Solution</summary> ```python import hashlib class ContentAddressableStore: def __init__(self): self._store = {} def store(self, data: bytes) -> str: content_hash = hashlib.sha256(data).hexdigest() if content_hash not in self._store: self._store[content_hash] = bytearray(data) return content_hash def retrieve(self, content_hash: str) -> bytes: if content_hash not in self._store: raise KeyError(f"Object {content_hash} not found in store") data = bytes(self._store[content_hash]) # Verify cryptographic integrity on read actual_hash = hashlib.sha256(data).hexdigest() if actual_hash != content_hash: raise ValueError(f"Integrity check failed: expected {content_hash}, got {actual_hash}") return data def exists(self, content_hash: str) -> bool: return content_hash in self._store # Verification cas = ContentAddressableStore() payload = b"IMMUTABLE_DATABASE_LEDGER_ENTRY" # 1. Deduplication hash1 = cas.store(payload) hash2 = cas.store(payload) assert hash1 == hash2 assert len(cas._store) == 1 # 2. Retrieval retrieved = cas.retrieve(hash1) assert retrieved == payload assert cas.exists(hash1) is True # 3. Tamper detection cas._store[hash1][0] ^= 0xFF # Simulate disk bit rot tamper_detected = False try: cas.retrieve(hash1) except ValueError: tamper_detected = True assert tamper_detected is True ``` </details>