Back to Notes

Design WhatsApp (Chat System)

1. Problem & Real-World Context

Design a real-time 1-1 and group messaging system: message delivery whether the recipient is online or not, delivery/read receipts, presence, and media sharing. The defining challenge isn't storage (that part is routine) — it's routing a message to whichever specific server instance currently holds the recipient's live connection, since WebSocket connections are stateful and don't fit the usual stateless-load-balancer model (see [[Communication Protocols]]).

Real numbers: ~2B users, ~100B messages/day — this is a genuinely enormous write volume, and the design must stay correct (no lost messages, correct ordering) under that load, not just fast.

2. Requirements

Functional: 1-1 messaging; group messaging (up to ~1024 members); delivery status (sent/delivered/read); online presence ("last seen"); media sharing (images, video, documents).

Non-functional: at-least-once delivery — messages must never be silently lost; ordering guaranteed within a single conversation; end-to-end encryption; low latency (<100ms delivery when both parties are online).

3. Capacity Estimation

  • Writes: 100B messages/day ÷ 86,400s ≈ ~1.16M messages/sec average (peak significantly higher around regional daytime overlap) — this alone rules out any single-instance relational store; the message store must be horizontally scalable from day one.
  • Storage (transient, not permanent): WhatsApp's actual model stores a message on the server only until delivered, then deletes it — the phone is the permanent store, not the server. This is a deliberate privacy + cost decision, not an oversight: if messages were retained server-side indefinitely, storage would grow unbounded at 100B/day.
  • Connections: 2B users, a meaningful fraction concurrently online — each held as a live WebSocket on some chat server. At, say, 100M concurrent connections and ~50K connections/server (a realistic ceiling given epoll-based I/O multiplexing — see [[Caching & Redis]]'s discussion of the same technique in Redis), that's ~2,000 chat server instances, each needing to know how to reach every other one for cross-server delivery.

4. API Design

Not a typical REST surface — mostly a persistent bidirectional channel (see [[Communication Protocols]]):

WS  /connect                                  — upgrade to WebSocket, authenticate
→   {type: "message", to: user_id, content, client_msg_id}
←   {type: "ack", message_id, client_msg_id}           — server confirms receipt ("sent ✓")
←   {type: "delivered", message_id}                     — recipient's device received it ("delivered ✓✓")
←   {type: "read", message_id}                          — recipient opened the conversation ("read ✓✓ blue")

client_msg_id is generated client-side before sending — the idempotency key that makes retries safe (§9).

5. High-Level Architecture

flowchart LR
    A[Alice's Client] -->|WebSocket| SA[Chat Server A]
    SA -->|Bob online,<br/>different server| RPS[Redis Pub/Sub<br/>msg:bob_id channel]
    RPS --> SB[Chat Server B]
    SB -->|WebSocket| B[Bob's Client]
    SA -->|Bob offline| MQ[Message Queue]
    MQ --> MS[(Message Store<br/>Cassandra)]
    MQ --> PUSH[Push Notification<br/>FCM/APNs]
    PUSH -.wakes.-> B
    B -.reconnects, fetches<br/>undelivered.-> MS

Walking a message (Bob online, different server): Alice's client sends over her WebSocket to Chat Server A → Server A assigns a Snowflake message_id and ACKs Alice immediately (client shows "sent ✓") → Server A looks up which server holds Bob's connection (Redis registry, §6) → publishes to a Redis pub/sub channel keyed to Bob → Bob's server (subscribed to that channel) receives it and delivers over Bob's live WebSocket → Bob's client sends a delivery receipt back through the same path in reverse → Alice's client shows "delivered ✓✓".

Walking a message (Bob offline): Server A can't find Bob in the connection registry → pushes the message onto a durable queue → persisted to the message store → a push notification (FCM/APNs) wakes Bob's device → on reconnect, Bob's client fetches all undelivered messages from the store.

6. Connection Management — the Core Routing Problem

Every chat server maintains many live WebSocket connections. The system needs a registry mapping user → which server instance currently holds their connection — this is the piece that makes cross-server delivery possible at all.

import redis
r = redis.Redis()

def on_connect(user_id: str, server_id: str) -> None:
    r.set(f"user:{user_id}:server", server_id, ex=60)   # short TTL — see below
    r.pubsub().subscribe(f"msg:{user_id}")               # this server now listens for Bob's messages

def on_disconnect(user_id: str) -> None:
    r.set(f"user:{user_id}:last_seen", int(time.time()))
    r.delete(f"user:{user_id}:server")

def route_message(sender_id: str, recipient_id: str, message: dict) -> None:
    target_server = r.get(f"user:{recipient_id}:server")
    if target_server is None:
        enqueue_for_offline_delivery(recipient_id, message)   # §5's offline path
        return
    r.publish(f"msg:{recipient_id}", json.dumps(message))     # recipient's server is subscribed

Why a short TTL on the registry entry (not just deletion on graceful disconnect): a server crash doesn't get a chance to run on_disconnect — without a TTL, a crashed server's users would appear permanently "connected" to a server that no longer exists, silently blackholing every message routed to them. A TTL refreshed on heartbeat means a crash self-heals within the TTL window instead of requiring a separate failure-detection mechanism (see [[Service Discovery & Health Checks]] for the general pattern).

7. Message Storage

Cassandra, partition key conversation_id, clustering key message_id (Snowflake, time-sortable — same technique as the News Feed problem's post IDs). This gives "fetch the last N messages in this conversation" as a single-partition, already-sorted scan — no secondary index or application-side sort needed. Write-heavy, append-only, horizontally scalable — the same reasoning as [[SQL vs NoSQL]]'s column-family recommendation for this access pattern.

Retention is deliberately short-lived, not permanent: a message is stored server-side only until the recipient's device confirms delivery, then deleted. This is a genuine design decision with real consequences: it caps storage growth despite 100B messages/day, and it's a meaningful privacy property (the server is never a long-term message archive) — worth stating as an explicit trade-off (permanent server-side storage would make "search my message history from the cloud" trivial, but WhatsApp's actual model doesn't offer that specifically because of this choice).

8. Group Messaging

Two different strategies depending on group size:

  • Small/medium groups (WhatsApp's actual approach, up to ~1024 members): fan-out on write — the message is stored once per recipient (or once, referenced by each recipient's per-user queue), the same push-heavy approach as the News Feed problem's non-celebrity path. At group sizes this small, the write amplification is bounded and acceptable.
  • Very large groups/broadcast-style (1000+ effective recipients): store one copy of the message, have each member's client reference/pull it rather than duplicating storage per member — trading some read-path complexity for a large storage-efficiency win. This mirrors the same push-vs-pull tension as the News Feed problem's celebrity case, applied to group size instead of follower count.

9. Delivery Guarantees & Idempotency

At-least-once delivery means retries can happen — the client resending a message it never got an ACK for (network blip, server crash mid-processing) must not create a duplicate on the recipient's side.

def handle_incoming_message(client_msg_id: str, sender_id: str, recipient_id: str, content: str) -> str:
    existing = message_store.get_by_client_id(sender_id, client_msg_id)
    if existing:
        return existing.message_id            # already processed — return the same ID, don't reprocess

    message_id = snowflake.next_id()
    message_store.save(message_id, client_msg_id, sender_id, recipient_id, content)
    route_message(sender_id, recipient_id, {"message_id": message_id, "content": content})
    return message_id

The client-generated client_msg_id is the idempotency key (same pattern as [[API Design Principles]]'s Idempotency-Key header for payments) — the server can always tell "have I seen this exact send attempt before" regardless of how many times the client retries.

10. Message Ordering

Within one conversation, Snowflake IDs (server-generated, time-based, with a per-machine sequence counter — see [[Design a Unique ID Generator]] for the full mechanism, and [[News Feed - Twitter Feed]] for the identical technique applied to posts) give a strict, collision-free ordering even across multiple chat servers, since sorting by ID is sorting by creation time. Clock skew between servers generating IDs independently is the subtlety: if server clocks drift, two messages sent moments apart on different servers could get IDs that don't perfectly reflect real-world send order. Mitigations: NTP-synchronized clocks tightly bounding drift, or Hybrid Logical Clocks (HLC) which combine a physical timestamp with a logical counter specifically to preserve causal ordering even under clock skew — worth naming HLC by name if pushed on "what if server clocks aren't perfectly synced."

11. Presence (Online Status / Last Seen)

HEARTBEAT_INTERVAL = 5    # seconds
ONLINE_THRESHOLD = 10     # seconds — allow for one missed heartbeat before flipping to offline

def on_heartbeat(user_id: str) -> None:
    r.set(f"user:{user_id}:last_seen", int(time.time()))

def get_presence(user_id: str) -> str:
    last_seen = r.get(f"user:{user_id}:last_seen")
    if last_seen and (time.time() - float(last_seen)) < ONLINE_THRESHOLD:
        return "online"
    return f"last seen {format_timestamp(last_seen)}"

ONLINE_THRESHOLD is deliberately a small multiple of HEARTBEAT_INTERVAL, not equal to it — a threshold exactly equal to the interval would flap a genuinely-online user to "offline" on every single heartbeat that arrives even a moment late. Privacy: users can opt to hide last_seen, which is a display-layer filter on top of this same underlying mechanism, not a different mechanism.

12. Media Sharing

Large files never flow through the chat server — that would turn a real-time messaging path into a file-transfer bottleneck. Instead: client uploads directly to blob storage via a pre-signed URL (see [[Blob & Object Storage]]), gets back a CDN-servable reference, and sends a normal-sized message containing just that reference. The recipient downloads from the [[CDN]], not from the chat server. For end-to-end encryption, the client encrypts the file before upload and includes the decryption key inside the (separately encrypted) message payload — the server and CDN only ever handle opaque encrypted bytes.

13. Failure Modes

  • Chat server crash: the client's WebSocket drops, it reconnects (to a different server via the load balancer), and on reconnect fetches any messages that were queued for offline delivery in the interim — the same offline path as §5, whether the "offline" reason was the recipient's device being off or their assigned server dying.
  • Message not ACKed within a timeout: client retries with the same client_msg_id — safe because of the idempotency handling in §9.
  • Redis pub/sub itself fails: cross-server real-time delivery degrades, but isn't a total outage — fall back to the recipient's client polling for new messages on reconnect/foreground, trading real-time delivery for eventual delivery until pub/sub recovers.

14. Interview Drill Questions

  1. Why does WhatsApp delete messages from the server after delivery instead of keeping a permanent history? — Two deliberate reasons stated together, not one: it bounds server-side storage growth at 100B messages/day (permanent retention at this volume is a very different, much larger cost problem), and it's a genuine privacy stance (the server is never a long-term archive of your conversations) — the phone is the source of truth for history, not the backend.
  2. Alice sends a message while Bob is reconnecting (his old server just crashed, new connection not yet registered). What happens? — The registry lookup in route_message finds no server for Bob (either the stale entry expired via TTL or was never re-registered yet), so the message correctly falls into the offline-delivery path — persisted and delivered once Bob's new connection registers, rather than being lost in a race between "technically online" and "registry not yet updated."
  3. How do you prevent a message being delivered twice if the recipient's server crashes right after delivering it but before sending the delivery-confirmation back to the sender's server? — This is the same idempotency mechanism from §9 viewed from the recipient's client side: the recipient's client can also dedupe by message_id on its own end, so even if the sender's side retries because it never got a delivery confirmation, redelivering the same message_id to an already-received client is a safe no-op.
  4. Design group messaging for a 100,000-member public broadcast channel — does the small-group fan-out approach still work? — No — fan-out-on-write to 100K members per message is the exact write-amplification problem the News Feed's celebrity case has; the correct approach here is the large-group strategy from §8 (single stored copy, referenced/pulled per member), explicitly naming that broadcast channels behave like celebrity accounts in a feed system, not like a small friend group.
  5. A user is a member of 50 active group chats. How does presence/read-receipt fan-out scale for them? — Read receipts are per-conversation, per-message — this doesn't multiply by group count the way message fan-out does; the actual scaling concern is each group's fan-out to its own members, not the user's total group membership count.

Cross-links

[[Communication Protocols]] · [[Caching & Redis]] · [[Message Queues & Kafka]] · [[Blob & Object Storage]] · [[CDN]] · [[API Design Principles]] · [[Service Discovery & Health Checks]] · [[News Feed - Twitter Feed]] · [[Design a Unique ID Generator]]