Lesson 001: Why One Server Is Never Enough
One box running your app and database is a real architecture with three hard ceilings: it gets slow, it fills up, and it disappears.
19 min read
System Design
One lesson a day, in order
A course in system design, written as a handbook: it starts from a single server and a single database and builds, one idea at a time, to designing whole systems at scale. Read it in order. Every lesson names the ones it stands on.
All lessons
Your progress
Loading your progress67 of 90 lessons published. One more usually arrives every morning. Progress is saved in this browser only.
Add this feed to any podcast app.
Phase 1
Foundations
Why one machine stops being enough, and the handful of ideas every distributed system is built from.
Done0 / 30
Lesson 002: Latency, Throughput and the Numbers Every Engineer Should Know
The handful of latency numbers worth memorising, why round trips cost more than bytes, and how to tell if a design is plausible before you build it.
16 min read
Lesson 003: What "Available" Actually Means: Nines, Downtime and Promises
Availability is a definition before it is a measurement: how to turn nines into hours, why downtime adds, and what redundancy cannot fix.
19 min read
Lesson 004: The Request's Path: DNS, TCP, TLS and HTTP in Plain Terms
What a browser actually does in the 180 milliseconds before your server hears a byte, and why every one of those steps can fail invisibly.
19 min read
Lesson 005: Vertical vs Horizontal Scaling: Bigger Box or More Boxes
A bigger machine is the cheapest fix and buys most products years of runway. The reason teams go horizontal anyway is almost never capacity.
17 min read
Lesson 006: Load Balancers: Spreading Traffic Without Losing Anyone
A balancer has to answer two questions about your fleet. The one about which boxes are alive is where the outages are, and almost nobody gets it right first.
20 min read
Lesson 007: Stateless Services and Where the State Actually Goes
Making a service stateless does not delete its state, it moves it, and each of the four places you can move it to sends you a different bill.
19 min read
Lesson 008: Caching: The Copy That Can Go Stale
A cache is a copy kept closer plus a rule for when to stop believing it, and the number that decides everything is what still reaches the database.
20 min read
Lesson 009: Cache Invalidation and the Cost of Being Wrong
Invalidation retires a cached entry when the data changes rather than when a clock runs out, and the TTL you keep is still what bounds how wrong you get.
20 min read
Lesson 010: Databases Under Load: Indexes, Slow Queries and the First Bottleneck
What actually makes a query cost what it costs, why an index can be present and useless, and the ratio to ask for before you read the SQL.
20 min read
Lesson 011: Read Replicas: Reading From a Copy
What streaming replication actually copies, why every replica does all of your writing, which reads are allowed to move, and the error that only happens on a copy.
20 min read
Lesson 012: Replication Lag: When Your Own Write Disappears
Where the delay between a primary and its copy comes from, how to measure it honestly, and four ways to stop a customer losing their own write.
20 min read
Lesson 013: Sharding: Cutting the Database So It Fits
What a shard is, the arithmetic of splitting one dataset across machines, why a sharded system always costs you the maximum rather than the average, and what stops working.
19 min read
Lesson 014: Choosing a Shard Key and Living With It
How to pick the column that decides where every row lives, the arithmetic of skew, and why this is the one Phase 1 decision you cannot undo cheaply.
17 min read
Lesson 015: Transactions and Isolation: What the Database Promises
What a transaction actually guarantees on one machine, why wrapping two statements in BEGIN does not stop an oversell, and what each isolation level really costs.
19 min read
Lesson 016: SQL, NoSQL and Blob Storage: Picking the Right Shelf
Why a store is a bundle of promises rather than a brand, what each shelf charges, and where Marlow's 216 gigabytes of cover images should have been living.
20 min read
Lesson 017: Message Queues: The Buffer With a Memory
What a queue actually buys, worked out on a wind farm backlog that took twenty two minutes to absorb eighty six seconds of work.
20 min read
Lesson 018: Asynchronous Work: Doing It Later Without Forgetting
A payout that never ran and nobody noticed for eleven days, the sum of a retry schedule nobody computed, and why absence is the one thing your instruments cannot see.
19 min read
Lesson 019: Idempotency: Safe to Retry
Seventeen card charges in December with no order behind them, and the difference between an operation that is atomic and one that is genuinely safe to run twice.
20 min read
Lesson 020: Timeouts, Retries and the Retry Storm
A payment provider had a bad thirty seconds and a ticketing service sold nothing for thirty nine minutes, and almost all of that was its own retry code.
20 min read
Lesson 021: Rate Limiting: Saying No Gracefully
A bookshop's rate limit sat idle for six months, then refused twelve hundred customers on its busiest morning and never once refused the robot it was written for.
20 min read
Lesson 022: CDNs and Edge Caching: Moving Bytes Closer
A bookshop put 1.2 million unchanging cover images behind a CDN with a one year expiry, measured a 62% hit rate, and found no dial attached to it.
20 min read
Lesson 023: Consistency, Intuitively: Strong, Eventual and Everything Between
What a reader is actually promised when there is more than one copy of a fact, why eventual promises less than it sounds, and how to choose the middle.
20 min read
Lesson 024: The CAP Trade-off Without the Theorem
What each side of a broken network should do while it cannot hear the other, why nobody sells you both answers, and why the choice is per operation.
20 min read
Lesson 025: Back-of-the-Envelope Estimation
How to get a usable number in four minutes from figures you already have, why the useful output is which term dominates, and when to distrust the answer.
20 min read
Lesson 026: Observability Basics: Logs, Metrics and Traces
What each of the three instruments throws away, why one label can delete a whole dashboard, and the number almost nobody draws that catches the outages the other numbers miss.
20 min read
Lesson 027: Failure Is Normal: Designing for Partial Outages
Why most outages are partial, how to choose in advance which features may disappear, and why a degraded mode nobody has ever run is a guess.
20 min read
Lesson 028: API Design: Contracts That Survive Change
What an interface actually promises, why adding is safe and changing is not, and why your oldest caller is usually your own cached page.
20 min read
Lesson 029: Authentication and Authorization at the Edge
Who you are and what you may do are different questions, and the cheapest place to ask the first one cannot answer the second.
19 min read
Lesson 030: Phase 1 Capstone: A Simple Web Service, End to End
A bookshop tried to draw itself on one page and found that a diagram records every box you bought and none of the things each one stopped being true.
20 min read
Phase 2
Mechanisms
How the building blocks actually work, where they break, and the trade-offs behind each choice.
Done0 / 30
Lesson 031: Consistent Hashing: Adding a Node Without Reshuffling Everything
Hashing a key modulo the machine count moves nearly everything the day the count changes, and the fix is older, cheaper and smaller than its reputation.
18 min read
Lesson 032: Leader Election: Why Someone Has to Be in Charge
Picking one machine to be in charge is the easy half; the hard half is what the loser is still allowed to do.
19 min read
Lesson 033: Consensus, Intuitively: Raft Without the Proofs
Lesson 011 already built a write quorum and left out the election rule, which is most of what Raft adds and all of what it is for.
21 min read
Lesson 034: Clocks, Ordering and Why "Before" Is Hard
A timestamp is a measurement with an error in it, so two stamps from two machines cannot be subtracted to learn which event happened first.
20 min read
Lesson 035: Distributed Locks and Their Sharp Edges
A lock promises you something about a resource, and almost every implementation of one actually promises you something about a client instead.
22 min read
Lesson 036: Exactly Once Is a Lie: Delivery Guarantees in Practice
A broker can only make exactly once true about what it owns, and your effect is not one of them, so you pick between duplicates and silent losses on purpose.
20 min read
Lesson 037: Event Streams: Kafka and the Log as a Database
A log keeps the record after you read it and gives each reader a position, so falling behind is a disk question and a mistake becomes a replay.
21 min read
Lesson 038: Event-Driven Architecture and Its Failure Modes
A producer that stops knowing who is listening also stops knowing what a replay will do, and every new consumer brings its own duplicate problem.
22 min read
Lesson 039: Sagas and Distributed Transactions
There is no rollback across machines, only a second operation that apologises, and the state it apologises for was visible to everybody while it lasted.
21 min read
Lesson 040: The Outbox Pattern: Writing to a Database and a Queue Safely
A message that was never written leaves no evidence, so write it in the same transaction as the fact and let a separate process publish it.
22 min read
Lesson 041: Backpressure: When the Consumer Cannot Keep Up
A queue with no bound still has one, and it is your heap; backpressure is choosing where the backlog waits and who gets told to stop.
22 min read
Lesson 042: Circuit Breakers, Bulkheads and Graceful Degradation
A breaker counts the far end's errors while you run out of your own resources, which is why the dependency failure that hurts most produces no errors at all.
21 min read
Lesson 043: Storage Engines: B-Trees vs LSM Trees
Eight thousand stock updates became a hundred and forty three megabytes of write ahead log, because the unit a disk writes is a page and never a row.
23 min read
Lesson 044: Object Storage Internals: How S3-Style Systems Work
Twenty seven abandoned multipart uploads cost a telemetry company a fifth of its backup bucket, because the bytes you pay for are not the bytes a listing shows.
22 min read
Lesson 045: Search: The Inverted Index
A bookshop that could not find the book called It, and what an inverted index really costs to read, to rank and above all to keep up to date.
22 min read
Lesson 046: Caching Patterns: Cache-Aside, Write-Through, Write-Behind
Three answers to one question, who writes the store and when, plus the bill nobody prices: a write-through resets the TTL, so partial coverage is worse than none.
22 min read
Lesson 047: Multi-Level Caches and Hot Keys
Consistent hashing sends one key to one node, so no sharding cools a hot key, and every fix moves its cost onto something you can buy more of.
22 min read
Lesson 048: Service Discovery and Configuration
A configuration outage leaves the running fleet untouched and breaks only the next process to start, so it gets collected one restart at a time.
22 min read
Lesson 049: API Gateways and Service Meshes
A gateway adds a hop to the request path and a sidecar adds a step to the startup path, and neither moves the state your policies actually need.
21 min read
Lesson 050: Multi-Region: Active-Passive, Active-Active and the Speed of Light
What a second region actually buys, why a cross-region write has a floor nobody can optimise, and why the failover you have never run is the whole risk.
22 min read
Lesson 051: Data Pipelines: Batch, Stream and Both at Once
Why the fast number and the careful number disagree, what a watermark really promises, and how to re-run a day without corrupting it.
22 min read
Lesson 052: Schema Evolution and Backward Compatibility
Why the migration that breaks nobody is the one that costs you, and why the deploy order is the part no tool can write for you.
22 min read
Lesson 053: Secrets, Keys and Rotation
Rotating a credential is five steps, only the last one helps, and almost nobody can prove they are allowed to run it.
22 min read
Lesson 054: Deployment Strategies: Blue-Green, Canary and Feature Flags
Every rollout strategy is a claim about what you would notice, and a canary extends the window in which two versions of your code have to agree.
22 min read
Lesson 055: Capacity Planning and Autoscaling
A scaler only adds the unit your bottleneck is counted in, and whether it keeps up is a race between its own lag and your load.
22 min read
Lesson 056: Cost as a Design Constraint
Sort a bill into rent, meters and somebody's afternoon before you read the total, because a loop in a template can cost as much as the whole fleet underneath it.
21 min read
Lesson 057: Security Boundaries: Trust, Zero Trust and Blast Radius
Measure a credential by what it can do that nothing else you hold can undo, because the key that may only write readings can still fabricate your product.
21 min read
Lesson 058: Offline-First Systems and Sync Conflicts
Once the edge of your system holds the only copy, it starts deciding for you: which readings to drop, and which of two orders for one book is real.
22 min read
Lesson 059: Audit Logs and Immutable Records
You cannot make a record unchangeable, only make a change detectable, so the question is never whether the rows can be edited but who would be able to tell.
22 min read
Lesson 060: Phase 2 Capstone: Hardening the Phase 1 Service
A bookshop closed its own checkout on purpose for two minutes, sold a book twenty two seconds later, and found the hole that costs it seven orders at Christmas.
21 min read
Phase 3
Design at scale
Whole systems, end to end: capacity, cost, failure, and the judgment to defend a design out loud.
Done0 / 7
Lesson 061: How to Run a System Design Interview, and a Design Review
An interview can only see what somebody had to compute, and a review can only see what somebody can check, which is why the refusals end up in neither.
20 min read
Lesson 062: Design a URL Shortener
A shortener is one sleepy database and two decisions you can never take back: the key space you spend as you grow, and the promise every link keeps forever.
24 min read
Lesson 063: Design a Distributed Rate Limiter
A limiter split across copies gets a caller's rate wrong in both directions, and the burst you already sold is what fixes it and sizes the counter tier.
24 min read
Lesson 064: Design a Key-Value Store
A key-value store's two irreversible decisions are both about conflicts: whether two nodes may accept one key's writes, and whether it can read the value to merge them.
25 min read
Lesson 065: Design a Chat System
A chat system's two irreversible decisions are who assigns a message's position and who owns the record of what you have seen. Only one makes a missing message visible.
25 min read
Lesson 066: Design a News Feed
A feed's irreversible decisions are what a copy of a post may hold and whether the list you served was a record. Neither is the fan-out question.
24 min read
Lesson 067: Design a Notification System
The push is an optimisation and the list is the guarantee, because a person can detect three separate failures here and the last hop belongs to somebody else.
23 min read
The examples
Three made-up systems, used in every lesson
None of these exist. They are invented for this course and reused from lesson to lesson on purpose, so their numbers become familiar and each new idea lands on ground you already know. When a lesson says Marlow Books, this is who it means.
Marlow Books
A small online bookshop
Four people, one server and one Postgres database. About 40 requests a second on a normal day and ten times that in the week before Christmas. The one box that the early lessons stress until it breaks.
Stagefront
An event ticketing service
Quiet most of the time, then a stadium show goes on sale at 10:00 and two hundred thousand people press the same button in the same minute. Oversold seats are a lawsuit, so correctness matters as much as speed.
Galewatch
Telemetry for wind farms
Nine hundred turbines, a reading every two seconds, over links that drop for hours in bad weather and come back with a backlog. Dashboards that lag by seconds, reports that scan a year.