Caching
Caching stores copies of data in a faster location to cut latency and load.
Where to cache
Browser -> CDN -> Load balancer -> App server (local cache)
-> Distributed cache (Redis / Memcached) -> Database
Strategies
| Pattern | Read path | Write path | Notes |
|---|---|---|---|
| Cache-aside (lazy) | App checks cache, on miss loads from DB and fills cache | Write to DB, invalidate or update cache | Most common |
| Read-through | Cache loads from DB on miss | n/a | Simpler app code |
| Write-through | n/a | Write to cache and DB together | Consistent, slower writes |
| Write-behind | n/a | Write to cache, flush to DB later | Fast, risk of data loss |
cache_aside.py
def get_user(user_id, cache, db):
key = f"user:{user_id}"
value = cache.get(key)
if value is not None:
return value # hit
value = db.fetch_user(user_id) # miss
cache.set(key, value, ttl=300)
return value
Eviction policies
LRU (least recently used), LFU (least frequently used), FIFO, TTL expiry.
lru.py
from collections import OrderedDict
class LRU:
def __init__(self, capacity):
self.cap, self.data = capacity, OrderedDict()
def get(self, key):
if key not in self.data:
return None
self.data.move_to_end(key)
return self.data[key]
def put(self, key, value):
self.data[key] = value
self.data.move_to_end(key)
if len(self.data) > self.cap:
self.data.popitem(last=False) # evict least recent
Hard problems
| Problem | Cause | Mitigation |
|---|---|---|
| Stale data | Cache not invalidated | Short TTLs, explicit invalidation, versioned keys |
| Cache stampede | Hot key expires and many requests hit the DB | Locking, request coalescing, jittered TTLs |
| Cache penetration | Requests for keys that never exist | Cache negative results, Bloom filter |
| Hot keys | One key gets huge traffic | Replicate the key, local caches |
NOTE
"There are only two hard things in computer science: cache invalidation and naming things." Plan invalidation before choosing what to cache.