Marlow Books is a four person online bookshop that exists only in this course, and in late September its founder fixed the logged-out-basket bug in about twenty minutes.
Lesson 005 had added a second server three weeks after the August resize, and within days customers were losing their baskets. Log in on box A, click a book, land on box B, and box B had never heard of you. The session lived in the application process's memory, which was a perfectly good place for it when there was one process.
The founder's fix was to stop keeping sessions on the server at all. Put the whole thing in a cookie: customer id, display name, and the basket, one line per title with its ISBN, its name, its price and a quantity. Serialise it to JSON, base64 it so it survives a header, set it on every response. Both boxes could read it. Both boxes agreed. The logouts stopped that afternoon and the founder went back to writing the Christmas email.
Two bills arrived, three months apart.
The first came in December, from a customer who wrote in to say she could not add a book to her basket. No error. She clicked, the page came back, and the book was not there. Every log on both boxes was clean, because nothing had gone wrong on either box. Browsers are only asked to keep 4,096 bytes per cookie and most of them stop exactly there. A basket line, once the title is in it, is around 120 bytes of JSON, and base64 grows anything by a third, so it costs about 160 bytes of the budget. Take out roughly 300 bytes for the customer's own fields and the cookie name, and the shop stops accepting books somewhere around the twenty third one. Past the limit the browser silently ignores the Set-Cookie header, keeps the copy it already has, and sends it back on the next request. The server reads a perfectly valid basket of twenty three books and renders it faithfully. There is no failure anywhere in the system. There is just a customer clicking a button that does nothing.
The second bill arrived in January, while the founder was reconciling December's card settlements against order totals. One order, eleven books, came out $47 under what the catalogue said those eleven books cost.
base64 is an encoding, not a lock. Anybody who looked at that cookie could read it, and anybody who could read it could change a price and hand it back. Since late September the shop had been quoting each customer their own prices and then charging them.
The founder deleted the cookie code in January, the same month the balancer from lesson 006 went in. The mistake was never base64. It was the assumption underneath it: that the state had to live somewhere both boxes could see, and a cookie was the only somewhere available.
There are four somewheres, and each one charges differently.
Which line stops the box being replaceable
State is anything a request handler remembers between requests. Not the row it read from the database, which it can read again, but the thing that only this process, or only this disk, currently knows.
A stateless service is one where any machine can answer any request, because everything the answer depends on is either in the request itself or in a store that every machine can reach. It does not mean the system has no state. Marlow's system is made almost entirely of state. It means none of that state is sitting inside the application tier, where it can be lost by a deploy, hidden by a balancer, or disagreed about by two boxes at once.
The test that actually works is a question you can ask of a single line of code. If this box vanished right now, mid-conversation, and a freshly booted copy took the next request, would the customer notice?
Here is a handler that fails it three times.
SESSIONS = {} # lives in this process
POPULAR = Counter() # so does this
def add_to_basket(request):
session = SESSIONS[request.cookies["sid"]]
session["basket"].append(request.form["isbn"])
POPULAR[request.form["isbn"]] += 1
return render("basket.html", session)
def upload_cover(request):
isbn = request.form["isbn"]
request.files["cover"].save(f"/var/marlow/covers/{isbn}.jpg")
return redirect(f"/book/{isbn}")
Line one is the logout bug: the session is a dictionary in this process and the next click may not come here. The save is lesson 005's cover upload, written to the local disk of whichever box served the form, so half the later requests for that image get a 404.
POPULAR is the interesting one, because it never breaks. It feeds the "moving this week" strip on the home page, and each box counts only the clicks it happened to receive. Nothing errors. Nothing times out. Two customers on two boxes just see different bestsellers, forever, and nobody files that as a bug because it does not look like one. Every alert stays green while the site quietly disagrees with itself. State like that, derived by the box from the traffic it happened to see rather than handed to it by a customer, is the shape that survives longest, because losing it is invisible.
Lesson 005 named three shapes of hidden state: a session in memory, an upload on local disk, a cron job baked into the image. That counter is the fourth, and it is the one that gets shipped.
There was a second way out of the logout bug and lesson 006 already refused it. A sticky session, where the balancer pins each customer to one box, leaves the state exactly where it is and makes one machine's survival that customer's problem. It is a real technique with two narrow uses and this is neither of them.
The four places, and what each charges
| Where it lives | Suits | Costs you |
|---|---|---|
| The client | small facts, cheaply refreshed | size on every request, no takebacks |
| A shared database | anything that must be right | a read, or worse a write |
| A shared cache | hot state you can rebuild | it is the only copy if you cannot |
| Object storage | anything file shaped | a second system, and dead links |
Nothing on that table is free, which is the part the phrase "just make it stateless" hides. You are not removing work. You are choosing which system does it and what happens when that system is having a bad morning.
The client can hold it, if you sign it
The founder's cookie was the right instinct executed with no defence. The defence has a name.
A signed token is a small blob of data the server gives the client, along with a signature computed over that data using a secret only the server knows. The client sends both back. The server recomputes the signature and compares. If they match, the data has not been edited, because editing it requires the secret. The usual recipe, HMAC with SHA-256, produces a 32 byte signature, 43 characters once it is base64url encoded, and verifying one costs a few microseconds. A session cookie built this way is typically 200 to 300 bytes: a customer id, an expiry, a claim or two, and the signature.
Signing fixes January's $47. It does not fix December's cookie limit, and it does not make the contents secret. Anyone can still read a signed token; they simply cannot change it. Put nothing in one you would mind a customer reading, and never put a price in one at all. The price belongs in the catalogue, and the basket should carry ISBNs and quantities so the server prices it at checkout from the row it owns.
Now the cost nobody mentions until it bites, which is that you cannot take a token back.
The whole appeal of a signed token is that no machine needs to look anything up. That is also its defect. Sack a contractor, ban an account, downgrade someone's role, and every token you already handed out keeps working until it expires, because nothing checks. If your expiry is 30 days, you have granted a banned account 30 more days. The usual answer is a short expiry, say fifteen minutes, plus a longer lived refresh token that gets checked against the database when it is used. Which means the honest description of a signed token is not "stateless authentication". It is a cached authorization decision, and the expiry is how long you are willing to be wrong.
That reframing turns a religious argument into arithmetic. Work out what the caching actually saves.
Christmas week peaks at 400 requests a second, and about 15% of those come from customers who are signed in, so 60 authenticated requests a second. Call it 8,000 people signed in and browsing at once. That is one request per person every 133 seconds. Over a fifteen minute token lifetime each person makes roughly seven requests, so a token turns seven database lookups into one: 60 a second becomes about 9.
Now price the thing you removed. A session lookup is a primary key read on a small table that lives entirely in memory, around 0.2 milliseconds. Sixty of those a second is 12 milliseconds of database time per second. Set that against what Postgres is already carrying at the same moment. Lesson 006 put the Christmas mix at about 4% full text searches, which is 16 a second at roughly 60 milliseconds each, so the searches alone are close to a full second of database work every second, before a single checkout. The lookups the token saves are about one percent of that.
So for Marlow the trade is: accept up to fifteen minutes of stale permissions, in exchange for one percent of a database's load. That is a bad trade, and the founder should keep the sessions table.
Reverse the traffic shape and the answer flips completely. A machine client hammering an API at five requests a second makes 4,500 calls inside the same fifteen minutes, and there the token is the difference between a system that works and a database that spends its whole life re-answering "yes, still them". The saving is exactly the number of requests per token lifetime. Count it before you argue about it.
The database can hold it, and it will charge you
Marlow's January fix was a sessions table: a session id, a customer id, an expiry, a small JSON blob for the basket. Every request with a session cookie reads one row by primary key.
About 40,000 live rows at 200 bytes each is 8 megabytes, which sits in Postgres's buffer pool and never touches the disk. It is one of the cheapest tables the shop has, and it fixed logouts, tampering and the cookie limit in one commit.
Then somebody asks for "keep me signed in as long as they are still shopping", and you write the line that updates last_seen on every request.
You have just converted every authenticated read into a write.
The CPU cost is small and the architectural cost is not. A write in Postgres leaves a dead version of the row behind for autovacuum to collect later, so a table of 40,000 live rows carries hundreds of thousands of dead ones through a busy day and its files grow well past the 8 megabytes the live data needs. Worse, it lands on the one Postgres that lessons 005 and 006 both named as the shop's ceiling, still on box A. And worst, a write can never be served by a read replica. Lesson 011 is about to offer Marlow the cheapest capacity it will ever buy, and a sliding session expiry quietly disqualifies every signed-in request from using it, before the replica even exists.
Use an absolute expiry and re-issue on the next natural write, or push the sliding timestamp into the cache, where a write costs a fraction of a millisecond, produces no dead rows and nobody is going to try replicating it.
The general rule is worth more than the example. Moving state to a shared store does not delete it. It centralises it, which is exactly what you wanted, and it reloads the ceiling that lesson 005 warned you about when it said that scaling out a stateless tier converts a compute limit into a database limit. Every piece of per-request state you move into Postgres is a request the database now sees that it did not see before. Three of those are free. Thirty are a capacity plan.
The cache can hold it, until it restarts
A shared cache, a Redis or a Memcached, is the natural home for session data: sub-millisecond reads, no vacuum, no replica question, and an expiry built into the store so cleanup is somebody else's problem. Lesson 008 owns how caches actually behave and lesson 009 owns what happens when they are wrong.
The question that belongs here is a different one. What is in this store that exists nowhere else?
A cached book page is a copy. Lose it and the next request rebuilds it from Postgres, slowly, and the customer sees a page. A session is not a copy of anything. If Marlow's cache restarts at 11 am on a Tuesday, 8,000 people are signed out mid-basket at the same instant, and no amount of retrying brings any of it back.
For a bookshop, that is survivable. Annoying, embarrassing, survivable. Write down that you decided it, because the same decision is fatal one system over.
Stagefront, this course's ticketing service, sells stadium seats that go on sale at exactly 10:00, when two hundred thousand people press the same button in the same minute. When you click a seat, Stagefront holds it for you for a few minutes while you find your card. That hold is state, it must be correct, and oversold seats are a lawsuit. If the holds live only in a cache and the cache restarts at 10:00:40, either every held seat is released and two people buy the same one, or the record of who was holding what is gone and thousands of seats stay locked with nobody able to buy them. Both endings are bad and one of them is a legal problem. Holds like that belong in the database with the seats, which is what lesson 015 is for.
The same shape shows up far from any data centre. Galewatch, which collects readings from nine hundred wind turbines, has turbines that buffer locally when a farm's link drops in bad weather and replay hours of readings when it comes back. For those hours the turbine is holding the only copy of that data in the world. It is state at the edge of the system, and it is exactly as lossy as the flash chip inside it. Lesson 058 has the sync problem; the point today is that "the only copy" is a property of the state, not of the store, and it is worth knowing which of your stores currently holds one.
Files go where files go
Marlow's covers were the second failure in that handler, and the fix is short because lessons 016 and 044 own the mechanism.
A cover written to /var/marlow/covers on box B exists on box B. Requests for it land on box A half the time, and no balancer saves you, because there is nothing here for a balancer to detect: both boxes are healthy, both answer in a couple of milliseconds, and only one of them has the file. The tempting fix is to sync the directory between the boxes every minute, and it is worse than the bug: you get two copies to keep in step, a race between the upload and the sync where the image 404s for up to a minute, and a scheme that has to be rewritten the day a third box appears.
Put files in object storage, which is a service whose whole contract is that any machine can put a named blob in and any machine can get it out. The application stores a key, not a path. Nothing about which box served the upload survives the request, which is the entire goal.
Lesson 001 already flagged the other reason: 1.2 million covers at 180 KB each is 216 GB, sitting on the same volume as the database. Moving them off is a state fix and a storage fix arriving in the same commit, which is rare enough to enjoy.
The cost is real. It is another system to operate, another dependency in lesson 003's chain, and a new class of bug where the row survives and the object does not, so your catalogue page renders a broken image forever. Deletes are the sharp edge: delete the row, fail to delete the object, and you are paying to store something nobody can find.
The job that has to run exactly once
The publisher payout job runs at 2 am on the first Sunday. Lesson 005 established what happened in October: the crontab was copied along with the machine image, so both boxes woke up, both loaded a year of order lines, both grouped by publisher, both emailed a CSV. Nobody was paid twice. The founder established that by spending Sunday morning proving it.
This is state too. The state is "has this month's payout already started", and it was living in nobody's head.
The four answers, cheapest first. Pin the job to one box and write down that you did. For four people this is a legitimate answer, and the price is that box A is special again, which is the property you spent two lessons removing, and if box A is down on the first Sunday nobody is paid and nothing tells you.
Take a lock. Both boxes wake up, both try to claim the run, the database decides who wins, because deciding who wins is the one thing a database is genuinely excellent at.
create table job_runs (
job_name text not null,
run_date date not null,
started_at timestamptz not null default now(),
primary key (job_name, run_date)
);
-- every box runs this at 02:00; exactly one gets a row back
insert into job_runs (job_name, run_date)
values ('publisher_payout', current_date)
on conflict do nothing
returning job_name;
If the insert returns a row, you own tonight's payout. If it returns nothing, somebody else does, so go back to sleep. That is the whole mechanism, it is ten lines, and it works on the database you already have.
Be clear about what it does not do. If the winner dies halfway through, the row is still there and no other box will ever pick the job up, so you have built at-most-once: the payout runs once or it does not run at all. Making it at-least-once means adding a finished_at and a lease that expires, and the moment a second box can retake an abandoned job you need the job to be safe to run twice, which is lesson 019 and is a harder problem than the lock. It also depends entirely on both boxes talking to one database. When there is more than one, the question becomes leader election, lesson 032, and scheduling it properly is lesson 076.
Four levels of answer for the same five minute bug, and the honest engineering is choosing the lowest one that survives your actual failures.
What is allowed to stay on the box
Not everything has to move, and a lesson that says it does is teaching you to build something slower than it needs to be.
Keep anything you can rebuild from a shared source without a customer noticing. Compiled templates. Connection pools, which are per box by definition and which lesson 006 showed you have to budget fleet wide anyway. Metrics counters that get scraped every ten seconds. A local in-memory cache of the catalogue, which lesson 047 will argue for properly.
The line is the one from the cache section. State you can rebuild is a copy, and copies on a box are fine. State that is the only copy is not, and it does not become fine because the box has been up for two hundred days.
One warning on the local caches, because it is the bug that eats an afternoon. Anything cached per box makes your boxes different from each other, and different boxes produce reports like "it only happens sometimes" and "it only happens for some people", which is the same sentence twice. A stale entry on box B and a fresh one on box A is a customer refreshing a page and watching the price change back and forth. When a bug is intermittent at exactly the rate your balancer splits traffic, stop reading the code and go and count your caches. Lesson 026 gives you the labels that make this visible in a graph rather than a hunch.
Recap
A stateless service is one whose state lives somewhere else. The state did not disappear. Marlow's sessions, covers and payout schedule all still exist; they moved out of the application tier and into stores that every box can reach, which is what makes any box able to answer any request.
Ask which line stops the box being replaceable. Read a handler one line at a time and ask whether a freshly booted copy could take the next request without the customer noticing. A dictionary in the process, a write to the local disk and a counter that only this box increments all fail, and only the first two ever show up as errors.
The token is a cached decision, and the expiry is how wrong you are willing to be. A signed token buys you a request that needs no lookup, and sells you the ability to revoke. Price the lookup you are avoiding first: Marlow's was one percent of what its database already does, and the same token in front of a chatty API client saves four thousand.
Moving state to a shared store centralises it. It does not delete it, so every per-request read you move into the database is load the database did not have, and every write you add there is a request that can never be served by a read replica.
The only copy. Whether a store may lose your data is a question about the data, not the store. A cached page is a copy and a session is not, so a cache restart at 11 am signs out eight thousand people, and the same design under Stagefront's seat holds is a lawsuit.
Whoever writes the row first runs the job. A unique key on the job name and the date turns "who owns tonight's payout" into an insert, and the database answers it correctly on the first try. It gives you at-most-once, which is worth knowing before you need at-least-once.
Check your understanding
Take the cookie the founder shipped in September and rebuild it properly. Say what stays in the cookie, what moves to the server, what gets signed, and what the checkout handler must recompute rather than trust. Then say whether either failure survives it: December's basket that silently stopped growing, and January's edited price.
A colleague proposes a fifteen minute signed token for an internal service where each caller makes about one request a minute. Work out how many database lookups the token saves per caller, and give the number you would want measured before agreeing.
Your session table has a
last_seencolumn updated on every request. Traffic is 200 authenticated requests a second. Describe what this does to the database over a day, and give two ways to keep sliding expiry without the writes landing there.A cache holding sessions is restarted during a deploy at 3 pm. List what breaks, who notices, and what you would have to change to make the same restart at 10:00:30 survivable for Stagefront's seat holds.
Marlow adds a third box. Walk through the payout job under each of the four answers above, and say at which point the
job_runsinsert stops being enough.
Next lesson
008 Caching: The Copy That Can Go Stale. Today's lesson put Marlow's state into shared stores and left every one of them a full request away; next lesson keeps a copy closer than that, which is the cheapest speed you can buy and the fastest way to serve a confident wrong answer.