Problem 06 — Design a Text-to-Image Generation Service

diffusion · async jobs · GPU queueing · content safety

Requirements

Prompt → 4 images at 1024×1024 within ~15 s. 5M images/day. Block unsafe prompts and outputs, watermark every image, keep 30 days.

How the model works (enough for interview)

Latent diffusion: text encoder → prompt embeddings; denoising network (U-Net or diffusion transformer) starts from random noise in a compressed latent space, removes noise over 20–50 steps guided by text; VAE decoder → pixels. Cost ∝ steps × resolution. Distilled models need 1–4 steps.

Estimates

5M images/day ≈ 58 images/s average, ~150 peak
Assume ~2 s of GPU time per image at 30 steps on a modern GPU
→ 150 × 2 = 300 GPU-seconds per second ≈ 300 GPUs at peak
Storage: 5M × ~1.5 MB × 30 days ≈ 225 TB in object storage

Architecture

flowchart LR
  U[Client] --> API[Job API]
  API --> PM[Prompt Safety]
  PM --> JQ[[Priority Job Queue]]
  JQ --> W[GPU Workers]
  W --> IS[Image Safety Classifier]
  IS --> WM[Watermark]
  WM --> OS[(Object Store + CDN)]
  API --> DB[(Job Status DB)]
  W --> DB
  U -->|poll or push| API

Deep dives

Async by design: return job ID immediately; client polls or gets a push. Show progress / low-res previews from intermediate steps. (Same pattern as [[Distributed Job Scheduler]].)
Queueing: separate priority lanes paid vs free. Workers pull batches with same resolution + step count to batch on GPU.
Quality vs speed: distilled few-step model for previews, full model for finals, optional upscaler stage.
Safety: prompt classifier before queueing (cheap); image classifier after generation (catches what prompts miss); blocklists for real people's names where policy requires.
Provenance: invisible watermark + C2PA content credentials.
Cost: spot/preemptible GPUs work well; jobs are short and retryable.

Evaluation

FID (realism), CLIP score (prompt alignment), human preference tests on a fixed prompt set (the one that matters most). Online: download rate, regenerate rate, safety-flag rate.

Likely follow-ups

Add editing / inpainting?
Personalized models (a user's face)?
GPU worker dies mid-job?

Related: [[Blob & Object Storage]] · [[CDN]] · [[index|GenAI index]]

Design a Text-to-Image Service