GenAI System Design — Index
GenAI System Design — Reading Order
The "design an LLM-powered system" round, at senior SDE level. You put a model inside a production system; you are not asked to train one. Original content written as a free alternative to ByteByteGo's Generative AI System Design Interview.
Interactive version: GenAI System Design Handbook (web). Same content as these notes, with diagrams and hidden-answer Q&A.
Before this folder: [[Learning/AI & ML/Karpathy - Deep Dive into LLMs]] (how LLMs work) and the classic SD basics in Concepts/.
Foundations (read in order)
| # | Page | Summary |
|---|---|---|
| 1 | [[GenAI SD Framework]] | 7-step plan for a 45-min round + what senior answers sound like |
| 2 | [[GenAI Vocabulary]] | ~45 terms in 6 groups: model, serving, RAG, adaptation, agents, safety/eval |
| 3 | [[GenAI Capacity Math]] | Weight memory, KV cache, decode speed, prefill FLOPs, traffic → GPUs, embedding storage |
| 4 | [[Vector Quantization]] | int8 / binary / PQ, IVF-PQ build + search in detail, tuning knobs, HNSW comparison, FAISS code |
Problems (worked designs)
| # | Page | Core hard part |
|---|---|---|
| 01 | [[Design ChatGPT-style Assistant]] | Streaming, context management, GPU cost, routing |
| 02 | [[Design Enterprise RAG Q&A]] | Ingestion, hybrid retrieval, ACL pre-filtering, citations |
| 03 | [[Design Smart Compose]] | <100 ms budget, small models, show-or-not threshold |
| 04 | [[Design Code Assistant]] | Fill-in-the-middle context, request cancellation, secrets |
| 05 | [[Design Support Agent with Tools]] | Agent loop, tool gateway auth, human approval, prompt injection |
| 06 | [[Design Text-to-Image Service]] | Diffusion, async GPU job queue, image safety |
| 07 | [[Design LLM Gateway]] | Token quotas, failover, semantic cache, cost attribution |
Older companion page: [[RAG & LLM System]] (HLD folder). Problem 02 goes deeper.
Practice
| Page | Summary |
|---|---|
| [[GenAI SD Question Bank]] | 14 short questions with model answers |
Free resources (priority order)
- Building A Generative AI Platform, Chip Huyen. Read first, ~1 h.
- Patterns for Building LLM-based Systems & Products, Eugene Yan. ~1.5 h.
- Building Effective AI Agents, Anthropic. ~30 min. Read before Problem 05.
- What We've Learned From A Year of Building with LLMs. ~2 h; skim the operations parts.
- chiphuyen/aie-book. Start with
chapter-summaries.md. - alirezadir/Machine-Learning-Interviews. Practice designs.
- FabG/genai-engineering-interview. Quick-fire revision.
- themanojdesai/genai-llm-ml-case-studies. Real company write-ups; browse 1–2 a week.
Paid, optional: ByteByteGo Generative AI System Design Interview, Chip Huyen AI Engineering (O'Reilly 2025).
13 pages.