Defining Nonfunctional Requirements

DDIA 2nd edition — Chapter 2 notes. (Complete — only book Summary left, skippable)

Sections covered: Functional vs Nonfunctional ✅ · Case Study: Home Timelines ✅ (Representing Users/Posts/Follows ✅ · Materializing & Updating Timelines ✅) · Describing Performance ✅ · Overloaded System Won't Recover ✅ · Latency vs Response Time ✅ · Average/Median/Percentiles ✅ · Use of Response Time Metrics (SLO/SLA, tail-latency amplification) ✅ · Reliability & Fault Tolerance ✅ (Faults/Failures · Fault Tolerance · Hardware/Software Faults · Humans & Reliability) · Scalability ✅ (Describing Load · Shared-Memory/Disk/Nothing · Principles) · Maintainability ✅ (Operability · Simplicity · Evolvability)

Remaining: Summary ⬜ (book recap — skip)

Functional vs Nonfunctional Requirements

• Functional requirements — what the application must do: the screens, buttons, and operations that fulfil the software's purpose.
• Nonfunctional requirements — the qualities the system must have: fast, reliable, scalable, legally compliant, secure, and easy to maintain.

This chapter uses one case study — a social network home timeline — to make these abstract qualities concrete.

Case Study: Social Network Home Timelines

Build X (Twitter): users post, and follow other users.

Load assumptions:

• 500 million posts/day ≈ 5,800 posts/sec on average.
• Spikes up to 150,000 posts/sec on big events.
• Average user follows 200 people and has 200 followers.

Representing Users, Posts, and Follows

Keep everything in a relational database: a users table, a posts table, and a follows table for relationships.

erDiagram
    users ||--o{ posts : "sends (sender_id)"
    users ||--o{ follows : "is follower"
    users ||--o{ follows : "is followee"
    users {
        int id PK
        string screen_name
        string profile_image
    }
    posts {
        int id PK
        int sender_id FK
        string text
        bigint timestamp
    }
    follows {
        int follower_id FK
        int followee_id FK
    }

The main read operation is the home timeline: recent posts by everyone the user follows. As SQL:

SELECT posts.*, users.* FROM posts
    JOIN follows ON posts.sender_id = follows.followee_id
    JOIN users   ON users.id = posts.sender_id
    WHERE follows.follower_id = current_user
    ORDER BY posts.timestamp DESC
    LIMIT 1000

This finds everyone current_user follows, fetches their recent posts, and sorts by timestamp for the 1,000 most recent.

Why the naive approach explodes:

• Posts must be timely — followers should see a post within ~5 seconds. Naive way: re-run the query every few seconds while the user is online (polling). At 10M users online, that's ~2M timeline queries/sec.
• The query itself is expensive: following 200 people means fetching + merging 200 senders' recent posts. 2M queries/sec × 200 = 400M lookups/sec — and that's the average; users following tens of thousands of accounts are far worse.

Materializing and Updating Timelines

Two fixes: (1) push instead of poll, and (2) precompute the timeline so reads hit a cache.

Materialized timeline — for each user, store their home timeline as a data structure. Every time someone posts, look up all their followers and insert the post into each follower's timeline — like delivering mail to a mailbox. Login = serve the precomputed timeline; new posts arrive via a subscription to the timeline stream.

• Cost moves to writes. Timelines are now derived data that must be updated on every post. When one request triggers many downstream requests, the multiplier is the fan-out factor.
• Fan-out math: 5,800 posts/sec × 200 followers ≈ 1M timeline writes/sec — a lot, but a huge win vs. the 400M read lookups/sec of the naive approach. (Read-heavy workload → pay on write.)
• Spikes can be queued. During a load spike, timeline deliveries can be enqueued and lag slightly; reads stay fast because they're served from cache.

This precompute-and-update pattern is materialization; the timeline cache is a materialized view — faster reads in exchange for more write work.

Extreme cases the materialized view must handle:

• User following huge numbers of high-volume accounts → very high write rate to their timeline. But they don't read it all, so it's OK to drop/sample some of their timeline writes.
• Celebrity with millions of followers posts → fanning out to millions of mailboxes is huge, and here dropping writes is not OK. Fix: handle celebrity posts separately — store them apart and merge at read time with the materialized timeline. Even so, celebrities require significant infrastructure.

Describing Performance

• Response time — time for a request to return its answer (seconds/ms/µs). Case-study examples: "time to load the home timeline," "time until a post reaches followers."
• Throughput — requests/sec (or data volume/sec) the system processes. For fixed hardware there's a maximum throughput. Case-study examples: "posts/sec," "timeline writes/sec."

Throughput and response time are coupled. Response time stays low while throughput is low, then rises as throughput climbs — because of queueing: an arriving request waits behind earlier requests the CPU is still processing. As throughput nears the hardware maximum, queueing delays rise sharply.

xychart-beta
    title "Response time vs throughput (queueing blows up near the limit)"
    x-axis "Throughput -->" 0 --> 100
    y-axis "Response time" 0 --> 100
    line [10, 11, 12, 14, 17, 22, 31, 48, 95]

When an Overloaded System Won't Recover

Pushed near its limit, a system can enter a vicious cycle and get more overloaded:

• Long queues → response times climb → clients time out and retry → request rate rises → worse. This is a retry storm.
• Even after load drops, the system can stay stuck overloaded until rebooted — a metastable failure, a cause of serious production outages.

Defenses:

• Client side — exponential backoff (increase + randomize the gap between retries); circuit breaker / token bucket to stop hammering a service that recently errored or timed out.
• Server side — detect approaching overload and proactively reject requests (load shedding), or tell clients to slow down (backpressure).

Latency and Response Time

• Response time — what the client sees; includes all delays anywhere in the system.
• Service time — the duration the service is actively processing the request.
• Queueing delay — waiting, not processing; happens at several points (waiting for a CPU, response packet buffered before send).
• Latency — catch-all for time a request is latent (not being actively processed). Network latency = time the request/response spends traveling the network.

flowchart LR
    A[Client:<br/>make request] -->|Network latency| B[Arrives at<br/>service]
    B -->|Queueing delay| C[Processing<br/>= service time]
    C -->|Queueing| D[Response<br/>ready]
    D -->|Network latency| E[Client:<br/>receive response]

Response time = network latency + queueing delay + service time + queueing + network latency. Service time is only the "Processing" box; everything else is latency. The whole span is what the client experiences.

Why response time varies request-to-request, even for an identical request: context switch to a background process, a lost packet + TCP retransmission, a GC pause, a page fault forcing a disk read, even mechanical vibration in the rack.

• Queueing delay is a big chunk of the variability. A server processes only a few things in parallel (e.g., limited by CPU cores), so a few slow requests hold up the ones behind — head-of-line blocking.
• Queueing delay is not part of service time → measure response time on the client side, or you'll miss it.

Average, Median, and Percentiles

Response time isn't one number — it's a distribution. Most requests are fast; occasional outliers are much slower. (Variation in network delay is jitter.)

Don't use the mean as your "typical" response time. The mean is useful for estimating throughput, but a few large outliers drag it up, so it doesn't represent what a typical user experiences. Use percentiles instead.

Sort all response times fastest→slowest:

The figure plots each request's response time as a bar, with the median, mean, p95, and p99 marked as horizontal lines — the mean line sits visibly above the median, showing the right skew.

Tail latencies (high percentiles like p99, p999) matter because they directly hit user experience:

• Amazon specifies internal-service response times at the 99.9th percentile — even though it's only 1 in 1,000 requests — because the slowest requests often belong to customers with the most data (and the most valuable accounts).
• But Amazon found optimizing p99.99 too expensive for too little benefit. Very high percentiles are hard to control — dominated by random events outside your control — with diminishing returns.

Use of Response Time Metrics

Tail-latency amplification — high percentiles matter most when one end-user request fans out to many backend calls in parallel. The user waits for the slowest call, not the average. So even a small fraction of slow backend calls makes a large fraction of end-user requests slow.

flowchart TB
    U([End-user request]) --> W[Web application]
    W -->|92 ms| B1[Backend 1]
    W -->|76 ms| B2[Backend 2]
    W -->|103 ms| B3[Backend 3]
    W -->|143 ms| B4[Backend 4]
    W -->|86 ms| B5[Backend 5]
    W -->|"487 ms ⟵ slowest"| B6[Backend 6]
    W -->|133 ms| B7[Backend 7]

One slow backend (487 ms) dominates: the end-user response time is ~487 ms even though six of seven backends answered in under 150 ms. Fan out to enough backends and almost every request hits at least one slow one.

• SLOs / SLAs — percentiles define expected performance/availability. A Service Level Objective (SLO) is a target — e.g., median < 200 ms, p99 < 1 s, and ≥ 99.9% of valid requests return non-error. A Service Level Agreement (SLA) is the contract specifying what happens (refunds, penalties) if the SLO is breached.
• Computing percentiles — naively: keep all response times in the time window and sort each minute. For efficiency, use approximate-percentile algorithms; open-source libraries include HdrHistogram, t-digest, OpenHistogram, and DDSketch.

Don't average percentiles. Averaging p99 values across machines or time windows is statistically meaningless — combine the underlying histograms instead.

Reliability and Fault Tolerance

Reliability = "continue working correctly, even when things go wrong."

• Fault — one part stops working correctly: a hard drive malfunctions, a machine crashes, a dependency has an outage.
• Failure — the system as a whole stops providing the required service to the user (i.e., it misses its SLO).

Key distinction: a fault is local; a failure is system-wide. Fault tolerance is the work of stopping faults from becoming failures.

Fault Tolerance

• A system is fault-tolerant if it keeps serving users despite certain faults occurring.
• If the system can't tolerate a part becoming faulty, that part is a single point of failure (SPOF).
• Always bounded — fault tolerance covers a certain number/kind of faults; tolerating all faults is impossible (if every node crashes, nothing can be done).
• Case study: if a machine doing fan-out (updating materialized timelines) crashes, another must take over without missing or duplicating any post.

Fault injection / chaos engineering — counterintuitively, it can help to increase the fault rate deliberately (e.g., randomly killing processes without warning). Chaos engineering builds confidence in fault-tolerance mechanisms by injecting faults in production-like conditions.

Hardware and Software Faults

Hardware faults (random and largely independent):

• Magnetic HDDs — ~2–5% fail/year → a 10,000-disk cluster expects ~1 disk failure/day.
• SSDs — ~0.5–1% fail/year; bit errors auto-corrected, but uncorrectable errors ~once/year/drive even when new (higher rate than HDDs).
• Other components — power supplies, RAID controllers, memory modules fail too, less often than disks.
• Bad CPU cores — ~1 in 1,000 machines has a core that occasionally computes the wrong result (manufacturing defects).
• RAM corruption — cosmic rays / physical defects; even with ECC, >1% of machines hit an uncorrectable error per year (usually crashes the machine). Pathological access patterns can flip bits on purpose (Rowhammer).
• Whole-datacenter loss — power outage, network misconfiguration, or permanent destruction.

At scale, hardware faults are frequent enough to be normal operation, not exceptional.

Tolerating hardware faults:

• First response: redundancy in individual components to lower the system's failure rate.
• Redundancy works best when faults are independent — but real component failures are often correlated (shared power, batch, environment).
• Single-machine redundancy raises one machine's uptime; a distributed system can tolerate a whole-datacenter outage. So cloud systems care less about individual-machine reliability and instead achieve high availability by tolerating faulty nodes in software. Availability zones mark which resources are physically co-located (and thus likely to fail together).

Software faults (systematic — correlated, hit many nodes at once):

• A bug that makes every node fail simultaneously under particular conditions (e.g., a bad input or a leap-second).
• A runaway process exhausting a shared resource (CPU, memory, disk, bandwidth, threads).
• A dependency slowing down, going unresponsive, or returning corrupted responses.
• Emergent behavior from interactions that don't appear when each system is tested in isolation.
• Cascading failures — one overloaded component drags down the next.

These bugs often lie dormant until triggered by unusual circumstances; there's no quick fix for systematic software faults.

Humans and Reliability

• Minimize the impact of human mistakes via: thorough testing (handwritten + property/fuzz testing), rollback for fast config revert, gradual rollouts of new code, clear monitoring + observability, and well-designed interfaces that make the right thing easy and the wrong thing hard.
• Blameless postmortems — after an incident, people share full detail without fear of punishment, so the whole org learns to prevent recurrence.

Scalability

Scalability = a system's ability to cope with increased load. It's not a one-dimensional label — saying "X is scalable" / "Y doesn't scale" is meaningless. Instead ask:

• If the system grows in a particular way, what are our options for coping?
• How can we add computing resources to handle the extra load?
• Given current growth projections, when do we hit the limits of the current architecture?

Describing Load

• First quantify current load before reasoning about growth. Load is usually a throughput measure: requests/sec, GB of new data/day, checkouts/hour.
• Other statistical characteristics of the load shape the access pattern and scaling needs: read:write ratio, cache hit rate, data items per user (e.g., followers).
• With load understood, probe what happens as it rises — two framings:
  • Resources fixed — increase load, hold CPU/memory/bandwidth constant: how does performance degrade?
  • Performance fixed — increase load: how much must you grow resources to hold performance steady?
• Goal: keep performance within the SLA while minimizing cost.

Linear scalability — double the resources → double the load handled at the same performance. Good. Often cost grows faster than linearly: e.g., with more data, a single write involves more work (index maintenance, contention) even though the request size is unchanged.

Shared-Memory, Shared-Disk, and Shared-Nothing Architectures

• Shared-memory (vertical scaling / scale up) — move to one bigger machine: more cores, RAM, disk. Threads/processes share the same RAM. Limits: cost grows super-linearly, and a top-end box still likely can't handle 2× the load; also a bigger blast radius (it's still one machine).
• Shared-disk — several machines with independent CPU/RAM, sharing data on a disk array over a fast network (NAS / SAN). Used for on-prem data warehousing, but contention + locking overhead cap scalability.
• Shared-nothing (horizontal scaling / scale out) — a distributed system of nodes, each with its own CPU, RAM, and disk, coordinating in software over a conventional network. Upside: potential to scale linearly, adjust hardware to load, and tolerate datacenter/region outages. (This is the architecture most of DDIA's later chapters assume.)

Principles for Scalability

• No one-size-fits-all scalable architecture.
• An architecture good for one load level won't survive 10×. Requirements evolve, so don't plan more than ~one order of magnitude ahead.
• Decompose into largely-independent components — the principle behind microservices, sharding, stream processing, and shared-nothing.
• Don't add complexity you don't need — if a single-machine DB works, don't go distributed; if load is predictable, prefer manual scaling over autoscaling.

Maintainability

Most of software's cost is not initial development but ongoing maintenance — fixing bugs, keeping it operational, investigating failures, adapting to new platforms/use cases, repaying tech debt, adding features. Three design qualities make maintenance cheaper:

• Operability — make it easy for ops to keep the system running smoothly.
• Simplicity — make it easy for new engineers to understand the system: well-understood, consistent patterns; no unnecessary complexity (accidental complexity).
• Evolvability — make it easy to change the system later for unanticipated use cases as requirements shift.

Operability: Making Life Easy for Operations

• At thousands-of-machines scale, manual maintenance is too expensive → automation is essential.
• But more automation ≠ better operability — it's a two-edged sword. There will always be edge cases (rare failure scenarios) needing manual intervention; find the sweet spot for your system and org.

A data system aids operability by:

• Monitoring + observability — expose key metrics and runtime-behavior insights.
• No dependency on individual machines — let machines be taken down for maintenance while the system keeps running.
• Good documentation + a clear operational model ("if I do X, Y will happen").
• Good defaults, but let admins override them when needed.
• Self-healing where appropriate, but with manual control available.
• Predictable behavior — minimize surprises.

Simplicity: Managing Complexity

• Complexity slows down everyone who touches the system and raises maintenance cost.
• Essential vs accidental complexity — essential is inherent in the problem domain; accidental arises only from limitations of our tooling/approach. Simplicity work targets the accidental kind.
• Abstraction is the best tool against complexity — hide implementation detail behind a clean, simple-to-understand facade.
• Build such abstractions with methodologies like design patterns and domain-driven design (DDD).

Evolvability: Making Change Easy

• Requirements change constantly; Agile patterns + tools (TDD, refactoring) help build under change.
• A system's ease of change is tightly linked to its simplicity and abstractions — loosely-coupled, simple systems modify more easily than tightly-coupled, complex ones.
• DDIA uses evolvability specifically for agility at the data-system level.

---

Interview Trade-Offs (Ch2)

Spoken-answer ammo. Each = a decision + the cost. Defend the pick out loud.

• State NFRs first → Before any design, name the nonfunctional targets: latency (p50/p99/tail), throughput, reliability, scalability, maintainability. Anchoring on "p99 < Xms at Y req/s" turns a vague design into a measurable one — senior signal.
• Fan-out on write vs fan-out on read → Read-heavy feed (timeline read 2M/s vs posts 5.8k/s)? Fan-out on write (materialize each follower's timeline) so reads are a cache hit. Cost: write amplification (~1M writes/s) + the celebrity hot-fan-out problem. Fan-out on read is cheaper to write but makes the hot read path expensive.
• Hybrid for celebrities → Don't fan-out celebrity posts to millions of mailboxes; store them separately and merge at read time. Cost: read path gets a merge step, but you avoid a write storm. Classic "it depends on the follower count" answer.
• Lossy optimization is OK sometimes → For a user following tens of thousands of high-volume accounts, drop/sample timeline writes — they won't read it all. Acceptable because the requirement is "good enough," not "every post." Naming where you can be lossy is a maturity signal.
• Percentiles, not mean → Report p50/p95/p99, never the average — outliers drag the mean up and hide the tail that users actually feel. Optimize the percentile tied to UX; stop before p99.99 (diminishing returns, uncontrollable randomness).
• Measure client-side → Queueing delay + network latency are invisible server-side. Measure response time at the client or you under-report what users experience (head-of-line blocking hides here).
• Keep throughput headroom → Response time vs throughput is a hockey stick — it spikes as you near the hardware max. Run with headroom; a system at 95% load has no room to absorb a spike.
• Protect against metastable failure → Near overload, retries cause a retry storm that keeps the system down even after load drops. Defend with exponential backoff + jitter, circuit breakers (client), load shedding + backpressure (server). Mention this the moment an interviewer says "what happens under a traffic spike?"
• Tail-latency amplification → If one user request fans out to N backends in parallel, the user waits for the slowest one — so per-backend p99 becomes the user's typical experience. Cost of fan-out: you must tame every backend's tail (hedged requests, tighter p99 SLOs), not just the average. Name this when a design fans a request across many shards/services.
• Fault vs failure → Senior framing: a fault is one part misbehaving; a failure is the system missing its SLO. The whole game is stopping faults from becoming failures. Identify the SPOF, then say how you remove it (redundancy, failover, replication). Note redundancy only helps when faults are independent — correlated failures (shared power/AZ) defeat it.
• Reliability: where to spend → Don't chase individual-machine reliability in the cloud — assume nodes fail and tolerate it in software (spread across availability zones). Single-machine redundancy buys uptime for one box; distribution buys survival of a whole-DC outage. Software faults are systematic (hit all nodes at once) so redundancy doesn't help them — testing, gradual rollout, and observability do.
• Vertical vs horizontal scaling → Scale up (shared-memory) first — simplest, no distributed-systems tax — until cost goes super-linear or one box can't hold 2× load. Then scale out (shared-nothing) for linear scaling + DC-level fault tolerance, paying coordination/complexity. Shared-disk (SAN/NAS) sits between but contention/locking caps it. "Don't distribute until you must."
• Don't over-engineer scale → Plan at most ~1 order of magnitude of growth ahead; an architecture tuned for 10× today is wasted complexity if requirements shift first. Prefer manual scaling for predictable load over autoscaling. Saying "I'd keep it on one Postgres until we hit X" is a stronger answer than premature sharding.
• Maintainability: kill accidental complexity → Separate essential complexity (inherent to the domain) from accidental (from your tooling/design) and attack only the accidental — via abstraction, consistent patterns, DDD. The cost of ignoring it: every future change slows down. Operability (monitoring, no per-machine dependency, predictable behavior) and evolvability (loose coupling) are design choices, not afterthoughts — name them when an interviewer asks "how does this evolve / who operates it?"

See [[synthesis/DDIA Interview Map]] for the full chapter → SD-topic map.

Next: Hands-On Exercise

Design drill — social-network home timeline (fan-out on read vs write)

Design X's home timeline. Lead with NFRs, then defend the fan-out choice out loud:

1. Fan-out on read or write? Justify with the load numbers (2M timeline reads/s vs 5.8k posts/s → read-heavy → fan-out on write). What's your write amplification?
2. How do you handle a celebrity with 50M followers? (Separate store + merge-at-read; why dropping writes is OK for over-followers but not for celebrities.)
3. A breaking-news event spikes posts to 150k/s — what gives? (Enqueue timeline deliveries, accept delivery lag, keep reads cache-fast; backpressure/load-shedding before the system goes metastable.)
4. A fan-out worker crashes mid-delivery — what breaks? (Reliability: the worker is a potential SPOF; another must take over without missing or duplicating posts. Idempotent delivery + checkpoint/queue offsets. Name fault vs failure.)
5. You've outgrown one timeline-cache box — how do you scale it? (Scalability: shared-nothing — shard timelines by user_id across nodes; what's the rebalancing/hot-shard story for a celebrity's followers?)

Hands-on — plot p99 vs throughput, watch the hockey stick (~1-2 hrs)

Goal: feel the response-time-vs-throughput curve and why tail latency is what matters.

1. Stand up any simple HTTP endpoint locally (Docker: a small Flask/Go service hitting Postgres, or even a static handler with a tiny sleep).
2. Load-test it at increasing concurrency with wrk, k6, or hey. Record throughput and p50/p95/p99 at each level.
3. Plot p99 vs throughput. Confirm the hockey stick: flat, then a sharp knee as you approach max throughput.
4. Find the knee — the throughput where p99 blows up. That's your usable capacity; everything past it is the queueing regime.
5. Done when: you have a p99-vs-throughput curve with the knee marked, and can say in one sentence why you'd run the service well left of it.

Log results in this folder (System Design/DDIA/) and drop the verdict + time spent in the current Weekly Tracker.