Consistency, Intuitively: Strong, Eventual and Everything Between

Marlow Books is a four person online bookshop that exists only in this course, and on a Tuesday in May it advertised a price it would not honour, in nine cities, for ten minutes, with nothing broken anywhere in the building.

The promotion was twenty percent off one title, 9780140449136, the same one that back in March served one signed-in customer's name and basket to the twenty odd strangers who shared his city. At 09:40 the founder ended it in the admin form. Eleven dollars twenty went back to fourteen. Lesson 009 built that write path and it works: the new price commits to Postgres on box A, and the same handler deletes the title's entries from the shared cache. Both halves ran.

At 09:58 a customer wrote in to ask why the checkout wanted two dollars eighty more than the page had said.

The founder loaded the book page. Fourteen dollars. They read the row. Fourteen dollars. They looked for the title's keys in the cache and found nothing, which is exactly what a correct delete leaves behind. It took twenty minutes to remember two things from lesson 022: that March had put the book page HTML behind a CDN with a ten minute expiry, and that the CDN's nine busy edges are simply the nine cities Marlow's customers live in.

Nine copies of that page. Each one laid down at the moment its own city first asked for it, each one expiring ten minutes after that moment, on its own clock, knowing nothing about an admin form back on box A.

09:40  price changes on box A, cache entries deleted
09:41  edge in city 3 expires, next asker gets $14.00
09:44  edge in city 7 expires
09:48  edge in city 1 expires
09:50  the last of the nine expires

So the wrongness did not end. It decayed. Lesson 022 measured this title at roughly a third of a request a second, which over those ten minutes is about two hundred page loads, and if the nine copies were spread evenly across the previous ten minutes, which nobody measured, about half of those two hundred were served a price the checkout would not honour. The founder's own browser was no help, because their nearest edge had already expired and the shop looked fine to them, which is lesson 006's Wednesday in November wearing different clothes.

Every component did its job. Postgres held one price, the cache held none, the CDN held the page it had been given for exactly as long as it had been told to. The question the founder kept asking, which of these is wrong, has no answer. The question with an answer is a different one: what had the shop promised the person reading that page?

That question is today.

Three different words spelled the same

Before anything useful can be said, two impostors have to leave the room.

Lesson 015 already evicted the first one. The C in ACID means the database's own rules hold across a transaction: constraints, foreign keys, the CHECK (stock >= 0) that Marlow spent years without. It is a promise about one copy obeying its own arithmetic, and it has nothing to do with today.

The second is durability, which lesson 016 priced as a separately sold promise. Durability says the bytes survive. Today's word says nothing about survival at all. A system can lose your data with perfect consistency, as long as every copy loses it in the same way.

What is left is the real one, and it only exists when there is more than one copy of a fact. It is the answer to a question a reader is asking without knowing they asked it: given that this fact lives in several places, what am I promised about the version I just got?

Note what that sentence does not contain. It says nothing about the data. The rows on box A are not consistent or inconsistent; they are just rows. Two shops with identical replication, identical caches and identical hardware can make completely different promises to their readers, purely by routing requests differently. Marlow made a promise about that price and broke it without changing a single byte of data anywhere.

Consistency is a property of what the reader is promised, not a property of the data. Almost every circular argument about consistency turns out to be two people describing two different readers.

Strong means behaving as though there were one copy

Start at the expensive end.

A system is strongly consistent when it behaves as though there were exactly one copy of the data and every operation happened one after another. A reader cannot tell how many machines you own. They cannot tell there is a network. Write, then read, and the read sees the write, whoever is asking and wherever they are standing.

That is a statement about behaviour, not about hardware. The precise name is linearizability, and it is not lesson 015's serializable: serializable is about whole transactions coming out in some order, linearizable is about one item's operations coming out in the order real time put them. For you, the behavioural sentence is the whole of it.

There are two honest ways to deliver it, and you have met both.

Read the copy that has everything. Lesson 011's routing audit sends Marlow's checkout guard to the primary, because the primary is the machine that has every committed write by definition. This costs no extra milliseconds. What it costs is the point of owning copies at all, and the arithmetic is unpleasant: lesson 008 measured 384 book page requests a second in Christmas week, and said plainly that without the cache absorbing them, 384 is 148 percent of the 260 a second that Marlow's pair of boxes can actually push at Postgres. "Make the book page strongly consistent" is therefore a sentence that means "buy more machines than the shop owns", and nobody at Marlow has ever phrased it that way.

Or make every copy wait. Lesson 011 priced this one: a commit that waits for the standby costs a network round trip, which is about 0.5 milliseconds inside one data centre and about 200 milliseconds from Mumbai to Virginia. Those two numbers deserve separate verdicts. For Marlow, whose replica sits beside box A and whose write rate is a couple of writes a second, half a millisecond on every commit is nothing at all. Across an ocean it is 200 milliseconds welded onto every checkout, which is two thirds of the card call lesson 015 measured at 300 milliseconds, paid on every single write so that a handful of reads can be answered somewhere else.

And lesson 012 left a correction here that catches everybody, so place it and move on: waiting for the standby in the default synchronous mode waits for bytes to reach the standby's disk, not for them to be applied. That buys durability. It does not buy visibility, and a write can be safe on the replica and invisible on it in the same instant. Only remote_apply buys the visibility, and it charges the writer for it.

Here is the part that is usually missed. You buy strong consistency per read, not per system. Marlow already has it on exactly the reads that need it, and has done since lesson 011: the stock guard, the customer's own orders straight after checkout. It does not have it on the book page, the search results, the publisher page or the review list, and that mix is not a compromise anybody is ashamed of. It is the correct answer, arrived at one read at a time.

What happens to all of this when the network between the copies breaks is lesson 024's problem.

Eventual is a promise about the end

Now the cheap end, and the term you will hear misused most often.

A system is eventually consistent when, for a given item, if updates to that item stop, every copy eventually returns the last value written. That is the whole guarantee. Read it twice, because the interesting part is everything it declines to say.

It says nothing about how long. It says nothing about what you see on the way. It does not promise that two readers see the same thing, or that you see your own write, or that what you see now is not older than what you saw a minute ago.

So the only useful question you can ask of a system described this way is: how long is eventually? And the word contains no answer. Numbers come from mechanisms you can point at. Lesson 012's replay rate gives a replica's bound, measured: a median of 6 milliseconds and a daily peak of 1.1 seconds when the 06:00 import lands. Lesson 008's TTL gives a shared cache's bound. Lesson 022's expiry gives an edge's. Lesson 009's delete on write gives a bound of however long a message takes to arrive, plus forever when the message is never sent. If you cannot name the mechanism that ends it, you do not have a bound. You have a hope with a technical name on it.

Marlow's bound that Tuesday was ten minutes, and the distribution inside it was nine independent clocks nobody had ever looked at.

Then there is the condition sitting at the front of the definition, which is where the word does most of its lying. Convergence is promised for an item whose updates have stopped.

Stagefront is a ticketing service that exists only in this course, where a stadium show goes on sale at exactly 10:00 and two hundred thousand people press the same button in the same minute. Lesson 008 caches its seat map for ten seconds. One seat row goes quiet almost immediately, since a seat is sold once. The map is not one row; it is a document derived from tens of thousands of them, and lesson 009 established that during an on-sale the rows underneath it change well over a hundred times a second. A derived view inherits the busiest of its ingredients, which is lesson 009's fastest changing ingredient rule pointed at convergence rather than at expiry.

So "the seat map is eventually consistent" is true, and it tells you precisely nothing about the ninety seconds that anyone cares about, because the map never reaches the quiet state the promise is about.

None of that is an argument against eventual consistency. Marlow's covers are eventually consistent and converged years ago, since they never change. It is an argument against quoting the phrase as though it were a service level. It is a shape, not a number.

The middle is a promise to one reader

Strong is a promise to everybody at once. Eventual is a promise to nobody in particular. Almost everything worth building sits between them, and the trick is that the useful promises in the middle are made to one reader rather than to the world.

That is why they are cheap. The world is large and a reader is one person with one session, and promising something to one person costs you a note in a cookie rather than a round trip on every commit.

Lesson 012 already built two of them and today only has to place them. Read-your-own-write says that once a write of yours is accepted, every later read by you sees it; it is the promise Nikhil lost when two of his three reviews came back missing. Monotonic reads says the world never gets younger: two reads by the same person must not go forward and then back, which is the promise a shared reader endpoint quietly withdraws, and which lesson 012 watched turn into a turbine reading 2.31 megawatts, then 2.28, then 2.31 again.

The third belongs to today. Monotonic writes says your own writes are applied in the order you made them. Marlow gets it free and has never noticed, because every write goes to one primary down one path, and one queue in one order is the definition of the thing. You lose it the day there are two places to write, which is lesson 013's shards or lesson 050's regions, or two paths to the same place. Picture the founder fixing a typo in a description, spotting a second one, and saving again thirty seconds later, with one of those saves going through lesson 018's queue and the other going direct. The later save lands first and the earlier one lands on top of it, so the description keeps the second typo and nothing appears in any log. Nobody built Marlow that way. Plenty of shops have built themselves that way one convenience at a time.

The fourth one matters because it is the one that breaks.

Every fix lesson 012 priced attaches the note to a session: a sticky window on a connection, a log position carried in a cookie, a client object that remembers. A person is not a session. Post a review from the phone on the train, open the laptop at home, and the promise you were given on the phone has no opinion about the browser you just opened. Read-your-own-write across devices is not a weaker version of the same promise. It is a different promise, about a human being, and the mechanism you bought was about a connection.

The general shape of those mechanisms has a name worth knowing. Lesson 012 gave the Postgres version, pg_current_wal_lsn on the write and pg_last_wal_replay_lsn on the read; generally it is a consistency token. The reader carries the highest version of the world they have seen, and the server refuses to answer from a copy behind it, either waiting for that copy to catch up or routing the read somewhere that has. It needs two things, and the second is the one that bites: a version number comparable across copies, and somewhere to keep the token that is not subject to the same lag as the data. It also turns a plain read into a read with a precondition, so the read can now wait, which means it can now time out, which is lesson 020's department and not free.

Last, the inversion. Lesson 012 found that the customers closest to you lose, because a short round trip means a short gap for the copy to win. Lesson 022 pointed out that an edge makes everybody close. Together they say something sharper than either: an edge has no idea who you are, so not one of the session promises survives it. There is nobody at the edge to carry a token, nobody to be sticky to, and no session for a note to live in.

Lesson 022's rule was to keep out of the cached document anything a person's own write is meant to change. May's Tuesday widened it. Keep out of the cached document anything that anybody's write is meant to change, because your own split does not survive the trip. Lesson 009 cached the page shell and read the price live from the row, and that split was real and correct and completely invisible to an edge, which stores what you sent rather than what you assembled. The fix Marlow shipped that week was to push the price out of the document, into the small second request lesson 022 had already invented for the customer's name and basket count.

One dial, at a different radius

Collect every staleness number this course has published and they turn out to be one dial.

The first three are copies Marlow owns; the fourth is the shape lesson 007 priced and turned down, and it is here because it is the same dial pointed at something that is not data. Four rows, four radiuses, one question underneath all of them: how far from the truth may a reader be, and for how long. A replica's is set by machinery, a TTL's by a person, an edge expiry's by a person and then quietly shortened by somebody else's eviction policy. Lesson 008's four places a copy can sit were a geography; read them again and they are a range of settings on one control.

Lesson 009 gave the model for choosing the number. Widen its first term from a missed invalidation to any chance of being read while wrong and it covers that whole table: the chance you are asked while wrong, times how long you stay wrong, times what being wrong costs per minute. Only the middle term belongs to you.

Run it on the hook. About two hundred requests arrived in the window, roughly half of them got the dead promotional price, the cost per wrong page is a refund or an argument, and the middle term was ten minutes. Ten minutes was not chosen against any of that. Ten minutes was chosen because it is a round number, which is how nearly every staleness figure in production got chosen, including most of yours.

The fourth row deserves a sentence of its own. A signed token is a cached decision: the permission was true at the moment it was signed, and the expiry is how much staleness you agreed to. Stale data and a stale decision fail differently. Stale data shows a wrong number on a screen, where a customer can see it and email you about it, which is what happened in May. A stale permission lets somebody do something they are no longer allowed to do, and there is no screen anywhere in the world on which that renders. You have to go looking.

And when you cannot make the copy fresh, lesson 012's answer still stands: make it dated. Galewatch, which collects a reading from each of nine hundred wind turbines every two seconds, learned this from a green tile sitting over a turbine that had been feathered since eleven that morning. Not a colour. The number, with its age beside it, ticking up when nothing new arrives.

The read that is really a write

There is a line where the consistency conversation stops helping, and a great deal of engineering effort gets spent on the wrong side of it.

Lesson 008 drew the first half. A cached read is a suggestion; a decision needs the row. Lesson 003 has the cautionary tale: a caching bug served a stale stock count for a day, every request a 200 in 8 milliseconds, and the shop sold 40 copies of a book it had 12 of.

The obvious lesson is that decisions need fresh data. Now watch lesson 015 take it away.

In July the shop ran a flash sale on twelve signed first editions. About ninety people pressed buy over twenty seconds. The shop took thirteen orders and the row finished at minus one. Every one of those thirteen buyers read the true row, by primary key, on the primary, in 0.2 milliseconds. Nobody read a cache. Nobody read a replica. Every one of those reads was as strong as a read is capable of being, and the shop still oversold.

The oversell had nothing to do with which copy anyone read. It came from the window between the read and the write, and the fix was one statement rather than one better copy: a conditional UPDATE ... WHERE stock > 0, which decides and writes in the same breath.

So: a read that decides is the first half of a write, which is lesson 011's third class of read that cannot move, and consistency is the wrong department for it. When somebody says "we need strong consistency here", the honest follow up is whether they need a fresher copy or an atomic decision. The first costs machines. The second is usually one line and is free.

Strong consistency is not useless there. It means the row you read is the row that exists, which is a real thing to own. It is simply not sufficient, and Marlow is the proof, because it had it.

So how stale may the on-sale page be

Lesson 022 ended with a question it refused to answer. Put s-maxage=1 on Stagefront's on-sale landing page and the origin sees one request a second per edge instead of its share of the 3,333 a second lesson 017 measured arriving. The page is then up to a second stale during a sale that lasts eighty seconds. Is that acceptable?

The vocabulary makes it answerable.

Ask who is promised what. Everybody gets the same document and nobody wrote anything, so read-your-own-write is not in play at all. Monotonic reads is, faintly: refresh twice just after the last seat goes and two edges a second apart can walk the show from sold out back to on sale. On a page nobody decides from, that is a flicker. The promise that matters here is how old the document may be.

Then price the second. Lesson 017 worked out that 40,000 seats at a purchase path converting 500 a second means the sale is over at 10:01:20. One second of staleness at the end of that is a page still saying "on sale" after it is not. Lesson 017's two hundred thousand all press inside the first minute, so at 10:01:20 that second catches stragglers rather than the flood. Sell a smaller stadium out at 10:00:30, with 3,333 a second still arriving, and the same second hands 3,333 people a page that is wrong about the show.

Now set it against what the business already accepts. The seat map inside the same sale is ten seconds stale by design, and lesson 008 called that correct. Ten is worse than one on exactly this dimension, by a factor of ten, and nobody has ever filed it as a bug. The reason is lesson 008's: clicking a seat does not buy it. The click goes to the database, which holds the seat or refuses.

So s-maxage=1 is right, and the reason is not that a second is a short time. It is right because nothing downstream of that page treats it as a decision. Change that and the answer changes with it: put "1,204 seats left" on the page, and the number stops being a suggestion, the second becomes a promise you cannot keep, and you are back to arguing about copies when what you needed was a statement that decides and writes at once.

Decide how stale a copy may be by asking what the reader is going to do with it, not by asking how old it feels.

Recap

Consistency is what the reader is promised, not a property of the data. Change nothing but your routing and you have changed what your customers are owed, which is how two engineers can stand in front of the same architecture and both be right about whether it is consistent.

Strong means behaving as though there were one copy. You get it by reading the copy that has everything, or by making every copy wait, and lesson 011's pair of numbers says which is survivable: half a millisecond next door, 200 milliseconds across an ocean, on every commit. You buy it per read and not per system, and Marlow has quietly owned it on two reads ever since.

Eventual is a promise about the end and nothing about the middle. It gives no bound, no promise about the way there, and its guarantee is stated about an item whose updates have stopped, which is a state Stagefront's seat map never reaches during the only minute that matters. "How long is eventually" is answered by a mechanism you can point at, or it is not answered.

The middle is a promise to one reader. Read-your-own-write, monotonic reads and monotonic writes are cheap because one reader is cheap, and a consistency token is the general way to buy them. A person is not a session, so the promise your phone was given does not follow you to your laptop, and it does not follow anybody to an edge.

One dial, at a different radius. A replica's lag, a TTL, an edge expiry and a token's lifetime are the same setting at different distances, and lesson 009's model chooses it: the chance you are asked while wrong, times how long you stay wrong, times what wrong costs. Only the middle term is yours, and it is usually a round number nobody argued about.

A read that decides is the first half of a write. Marlow oversold from the primary, at 0.2 milliseconds, with the freshest read available anywhere in the shop. When freshness is not the problem, a better copy is not the fix.

Check your understanding

1. A colleague says your service is "eventually consistent, so we are fine". Write down the three things that sentence does not promise, and the one question you would ask to turn it into a number.

2. Marlow's founder proposes fixing May's Tuesday by cutting the edge TTL from ten minutes to thirty seconds. Estimate what that changes using lesson 009's cost of being wrong model, say what it costs at the origin using lesson 022's arithmetic, and say whether you would do it instead of the fix the shop shipped.

3. A customer service tool shows an order as pending when the order was paid four seconds ago, and it reads from a replica. Say which promise is being broken, for which reader, and give two fixes at different prices. Then say which one you would pick if the tool were used by a person on the phone to an angry customer.

4. Your team wants to add "you may still be able to buy this" to a page cached at the edge for sixty seconds. Argue both sides using today's vocabulary, and describe the version of that feature you would actually ship.

5. Galewatch wants engineers at a site office to see a turbine's output with no more than two seconds of staleness, and the site link has a 45 millisecond round trip that drops for hours in bad weather. Say which of today's promises you can offer and which you cannot, and what you would put on the screen when you cannot.

Next lesson

024 The CAP Trade-off Without the Theorem. Today priced consistency while the network was working; next lesson takes the network away in the middle of a write and asks which of the two promises you would rather break.