Back to Notes

Build Your Own Redis — Topics to Learn

Hands-on companion to [[../Redis|Redis notes]]. Goal: understand Redis internals by building a minimal clone. You write the code — this is the list of concepts to go through per stage.

Scope discipline (job-switch alignment): Redis-playlist hands-on, not an open-ended side project (kill list). Time-box ~4–6h total. Do Stages 1–4. Stages 5–6 only if time.

Language: Python — selectors models Redis's single-thread event loop one-to-one (epoll/kqueue). Go's goroutine-per-conn is the opposite model, off-plan for this sprint.

Prerequisites (read first)

  • TCP socket lifecycle — socket → bind → listen → accept → recv/send → close
  • Blocking vs non-blocking I/O — why recv() blocks; what non-blocking returns
  • I/O multiplexing — select / poll / epoll (Linux) / kqueue (macOS); one thread watching many sockets
  • Python selectors module — the high-level wrapper over the above
  • Event loop mental model — select-ready → dispatch → repeat
  • RESP framing — see [[../Redis#Talking the Redis Language — RESP]]

Stage 1 — TCP server, single connection

  • socket module: creating, binding, listening, accepting
  • SO_REUSEADDR and why
  • recv/send loop + connection close handling
  • Test with nc localhost 6379

Stage 2 — RESP encode / decode

  • Parse the 5 RESP types by prefix: + simple, : int, $ bulk, * array, - error
  • Read framing: length-prefix + CRLF delimiters
  • Decode a command = array of bulk strings → ["PING"]
  • Encode replies: simple string, bulk string, null ($-1), error
  • (later) handling partial reads — a command may not arrive in one recv

Stage 3 — PING → PONG

  • Command dispatch (normalize case, switch on cmd[0])
  • PING with and without argument
  • Test: redis-cli -p 6379 PING → PONG

Stage 4 — I/O multiplexing (many connections) ← core lesson

  • selectors.DefaultSelector(), register/unregister, EVENT_READ
  • Distinguish listener-ready (→ accept new conn) vs client-ready (→ read)
  • Non-blocking sockets in the loop
  • Clean disconnect handling (unregister + close)
  • Understand: single thread, no locks — this is the Redis design
  • Test: 3 concurrent redis-cli sessions

Stage 5 — SET / GET (optional)

  • dict as keyspace
  • SET k v → +OK; GET k → bulk or null
  • Why no locking is needed (single-threaded insight made concrete)

Stage 6 — TTL / expiry (optional)

  • SET k v EX <sec> parsing
  • Store expiry timestamp
  • Lazy expiry (check on access) vs active sweep

Done-when

  • redis-cli PING → PONG
  • Multiple clients served on one thread
  • (stretch) SET/GET work
  • Note one thing that surprised you back in [[../Redis]]