Load Balancing
What & Why
Distributes incoming requests across multiple servers — prevents any single instance from being overloaded, enables adding/removing capacity without downtime, and is the mechanism that makes horizontal scaling actually usable.
flowchart LR
C1[Client] --> LB{Load Balancer}
C2[Client] --> LB
C3[Client] --> LB
LB --> S1[Server 1]
LB --> S2[Server 2]
LB --> S3[Server 3]
Static Algorithms (ignore current server state)
| Algorithm | Mechanism | Weakness |
|---|---|---|
| Round Robin | Cycle through servers in fixed order | Assumes all servers/requests are equal-cost — a slow request or weak server still gets its "turn" |
| Weighted Round Robin | Requests proportional to an assigned capacity weight | Static weight doesn't reflect real-time load |
| Consistent Hashing | Hash client/request key onto a ring, route to nearest server clockwise | See [[Consistent Hashing]] for the full mechanism + virtual nodes |
| Geo-based | Route by client location to the nearest regional cluster | Doesn't rebalance for skewed regional load |
Dynamic Algorithms (consider live server state — more accurate, more overhead)
| Algorithm | Mechanism | Weakness |
|---|---|---|
| Least Connections | Route to the server with fewest active connections | Ignores server capacity differences |
| Weighted Least Connections | Least connections, tie-broken by capacity weight | More complex to compute per-request |
| Least Response Time | Route to whichever server is currently responding fastest | Needs continuous latency measurement, more overhead |
Why Simple Hashing Fails (motivates consistent hashing)
server = H(key) % N. Works — until N changes (a server is added or removed), at which point nearly every key remaps to a different server. On average, remappings ≈ number of requests. That cache-invalidation storm is exactly what consistent hashing exists to avoid — full ring + virtual-node mechanism in [[Consistent Hashing]].
Sticky vs. Non-Sticky Sessions
- Sticky: all of a client's requests go to the same physical server — needed if session state lives in that server's memory.
- Non-sticky: any server can serve any request — requires session state externalized (Redis, shared DB). More scalable (any server can die without losing sessions) and the default in stateless-service architectures.
Interview Drill
Q: You add 3 more servers to a system using simple modulo hashing for load balancing. What breaks?
A: H(x) % N changes for almost all x when N changes, so nearly every key remaps to a different server — a mass cache-invalidation storm. This is the textbook motivation for consistent hashing.
Cross-links
[[Consistent Hashing]] · [[Proxies (Forward vs Reverse)]] · [[Database Replication & Sharding]]