Caching & Redis

Caching appears in virtually every system design interview. Redis is the default distributed cache. Master both the concept and the trade-offs.

Why Cache?

• DB reads are ~1–10ms. Cache reads are ~0.1ms.
• Reduces load on the DB — the same DB can handle 10x traffic with a cache in front.
• Absorbs read spikes without scaling the storage layer.

Cache hierarchy (fastest → slowest): CPU registers → L1/L2/L3 → RAM → SSD → Network (Redis) → HDD → DB.

Cache Strategies

Cache-Aside (Lazy Loading) — most common

Read:  check cache → miss → read DB → write to cache → return
Write: write DB → invalidate (or update) cache

def get_user(user_id):
    cached = redis.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)          # cache hit

    user = db.query("SELECT * FROM users WHERE id = ?", user_id)
    redis.setex(f"user:{user_id}", 3600, json.dumps(user))  # TTL 1hr
    return user

def update_user(user_id, data):
    db.update("UPDATE users SET ...", user_id, data)
    redis.delete(f"user:{user_id}")        # invalidate

Pros: cache only what's needed, DB is source of truth, cache failure = degraded not broken. Cons: cache-miss penalty — first request is slow (cold start).

Write-Through

Write DB and cache simultaneously; read is always in cache if not expired. Pros: cache always consistent with DB. Cons: write latency includes the cache write; write amplification for data rarely read.

Write-Behind (Write-Back)

Write cache → async flush to DB later (batched). Pros: very low write latency. Cons: data loss risk if cache crashes before DB flush; added complexity.

Read-Through

App reads from cache only; cache is responsible for loading from DB on miss. Common in managed caches (ElastiCache DAX for DynamoDB).

Eviction Policies

Redis default: allkeys-lru (evict any key, LRU) or volatile-lru (only TTL-set keys).

Cache Invalidation & Stampede

Cache invalidation is one of the hardest problems in CS. Two strategies: TTL-based (accept staleness up to TTL, simple) or event-driven (delete/update cache key on DB write, consistent but more moving parts).

Cache stampede (thundering herd): a popular cached item expires, and many concurrent requests miss simultaneously, flooding the DB. Fix with a mutex on cache miss so only one process recomputes:

import time

def get_with_lock(key, ttl):
    val = redis.get(key)
    if val:
        return val

    lock = redis.set(f"lock:{key}", 1, nx=True, ex=30)  # nx = set only if not exists
    if lock:
        val = db.query(...)
        redis.setex(key, ttl, val)
        redis.delete(f"lock:{key}")
        return val
    else:
        time.sleep(0.1)  # wait for lock holder
        return redis.get(key)

Redis Data Structures

Redis for Rate Limiting

def is_rate_limited(user_id: str, limit: int, window_seconds: int) -> bool:
    key = f"rate:{user_id}:{int(time.time() // window_seconds)}"
    count = redis.incr(key)
    if count == 1:
        redis.expire(key, window_seconds)
    return count > limit

Sliding window variant: use a sorted set, score = timestamp. See [[Rate Limiter]].

Redis Persistence

Pure cache (DB is source of truth) → no persistence or RDB only. Session store / rate-limiter counters (data loss = bad) → AOF everysec.

Redis Replication & Clustering

Master-Replica: one master (writes), N replicas (reads), automatic failover via Redis Sentinel. Redis Cluster: hash slots 0–16383 split across nodes (~16383/N per node). Key user:123 → hash → slot → correct node; client needs a cluster-aware library.

Why Redis Is Fast: Single-Threaded + I/O Multiplexing

Single-threaded — one command at a time, one thread. Eliminates race conditions (no locks needed for individual commands) and lock-contention variance. Critical implication: slow commands block everything.

KEYS *                          # freezes the whole server on large keyspaces — never in production
SCAN 0 MATCH "user:*" COUNT 100 # non-blocking iteration — use this instead

I/O multiplexing (epoll/kqueue) lets one thread handle thousands of concurrent connections: the OS notifies Redis when data arrives on any socket; Redis processes it (sub-ms) and moves to the next ready socket. Scales to ~100K concurrent connections on modern hardware, with zero per-connection thread overhead.

Data Structure Internals (depth for follow-ups)

• Strings: 3 encodings — int (small integers, no parsing), embstr (<44 bytes, single allocation), raw (long strings, separate allocation).
• Lists: doubly-linked (or ziplist when small). LPUSH/RPOP are O(1); LINDEX list 50000 is O(N) — never access list middle elements at scale.
• Sorted Sets: skip list + hash table dual storage — O(log N) for ZADD/ZRANK/ZRANGEBYSCORE, O(1) for ZSCORE. Each member is stored twice — a 10M-member leaderboard costs ~500MB+ RAM.
• Hashes: small hashes (<512 fields, values <64 bytes) use ziplist encoding — contiguous memory, no pointer overhead, roughly 60–70% memory savings versus one key per field.

Lua Scripts — True Atomicity

Multi-step atomic operations (rate limiting, distributed locks, dedup) need Lua, not MULTI/EXEC (which does not roll back on error — see below).

-- rate_limit.lua: atomic check-and-increment
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local ttl   = tonumber(ARGV[2])

local current = redis.call('GET', key)
if current == false then current = 0 else current = tonumber(current) end

if current < limit then
    redis.call('INCR', key)
    redis.call('EXPIRE', key, ttl)
    return 1   -- allowed
else
    return 0   -- rate limited
end

The entire script executes atomically — no interleaving possible. Cluster note: all keys in a Lua script must be in the same hash slot — use hash tags ({user}:1, {user}:2) to guarantee co-location.

Transactions Gotcha — No Rollback

MULTI/EXEC guarantees commands execute in order, atomically (no interleaving) — but does not roll back on error. If one command in the block fails (e.g., INCR on a string value), the others still execute. For true all-or-nothing atomicity, use Lua.

Pipelining — Bulk Throughput

# Without pipelining: 1000 network round trips ≈ 1 second
for i in range(1000):
    redis.set(f"key:{i}", f"value:{i}")

# With pipelining: 1 round trip ≈ 10ms
pipe = redis.pipeline()
for i in range(1000):
    pipe.set(f"key:{i}", f"value:{i}")
pipe.execute()

Pipelined commands are not transactional — they execute independently, just batched over the wire. For atomicity, use Lua.

Monitoring

SLOWLOG GET 10          # last 10 slow commands (default threshold: 10ms)
INFO memory             # used_memory, mem_fragmentation_ratio
INFO stats              # keyspace_hits, keyspace_misses → cache hit rate
INFO replication         # master/replica lag

Alert thresholds worth naming: memory > 80% of maxmemory, cache hit rate < 95%, unexpected evicted_keys > 0, replication lag > 1s.

Interview Trade-off Table

Standard interview answer structure: "I'd use cache-aside with LRU eviction and a 5-minute TTL. For writes, invalidate the cache key immediately. For read-heavy data like user profiles, this gives ~95% cache hit rate at steady state."

Interview Drill

Q: Your rate limiter does GET → check → INCR → EXPIRE as four separate Redis calls. What's wrong? A: A race condition — two concurrent requests can both GET the same pre-increment value, both pass the check, and both increment, silently admitting one more request than the limit allows. Fix: collapse the check-and-increment into a single Lua script so it executes as one atomic unit with no interleaving window.

Cross-links

[[CAP Theorem & Consistency Models]] · [[Rate Limiter]] · [[Message Queues & Kafka]] · [[Consistent Hashing]] · [[Database Replication & Sharding]]