Offline-First Systems and Sync Conflicts

Galewatch is a telemetry company that exists only in this course. It takes a reading every two seconds from each wind turbine on the farms it monitors and sells the owners dashboards. On Monday 26 October 2026 one of its engineers ran a query nobody had needed before, and sixty turbines came back with the same answer.

The weekend had been weather. Tarrow Ridge is the farm lesson 002 measured: sixty turbines along a ridge, a site office in a portable building at the bottom of a long access road, and a fixed wireless link doing about 20 megabits a second. The link went down at 06:20 on the Saturday and came back at ten to ten on the Sunday, which is twenty seven and a half hours.

Nobody worried, and this course is the reason. Lesson 003 settled it: when a farm's link drops, the turbines hold their readings in flash and replay them when it returns, so availability breaks and durability holds. Lesson 017 sat through four hours of that and did the division, 432,000 readings off those sixty turbines, every one delivered. Lesson 041 reported seven and a half hours of it at a bigger farm in August. Lesson 017 also named the mechanism in a clause, and lesson 046 the reason it deserves that reputation: a turbine deletes its local copy only once an ingest box has answered 200.

On the Monday the farm's owner asked for the weekend. A blade had been damaged and their insurer wanted the two second record, which is the thing Galewatch sells.

Tarrow Ridge's Saturday ends at 15:26.

Every turbine had stored 16,384 readings and stopped. Not about sixteen thousand. The same 16,384 on all sixty of them, which is nine hours and six minutes of weather each, and then nothing at all until the link came back and live readings resumed. The turbine clocks are not synchronised, which is lesson 034's subject, so the sixty holes do not begin at the same second. The counts do.

Twenty seven and a half hours is 99,000 seconds, so each turbine took 49,500 readings while it was cut off. 16,384 came home. 33,116 did not, times sixty, which is 1,986,960 readings that exist nowhere.

A storm does not stop on a power of two.

What a full log can do, and how to tell which it did

Lesson 041 walked Galewatch's chain of buffers from the database up to the wind and left a question at the top of it. The flash in a turbine is a queue with a producer, a consumer, a depth and a drain rate, and no graph anybody in the office can see. It held four hours in lesson 017 and seven and a half in August. Nobody knew what it did in the eighth hour, because the firmware author picked one of four behaviours and nobody had read that code.

Here are the four. A fixed-size store that is full can overwrite the oldest thing it holds, refuse the new thing, stop producing new things at all, or fall over in a way nobody chose.

Out loud, because a table is the one thing the audio version will not read you: one of the four keeps the end of the outage, two keep the beginning, one keeps almost nothing, and the only difference between the middle two is whether the turbine also stopped doing its day job.

And the choice between them is a product decision wearing a firmware costume. A dashboard wants the end of the outage, which is the reading from five minutes ago. A warranty claim, and lesson 051's rollup behind it, want continuity, which is the beginning. One turbine cannot know which, so whoever wrote those four lines answered for everybody.

Which gives you something better than source code. The position of the hole names the policy. Tarrow Ridge came back holding the beginning of the storm and none of the end, so its firmware is one of the two that keep the beginning rather than the one that overwrites. You can read that off the data from a desk, in one query, without the file.

What the data will not separate is refusing a write from stopping the sampler, because both leave the same rows behind. Somebody did finally open the file, which took an afternoon and settled that last half. The append walks a slot index to the end of a 16,384 record region, returns an error, and the caller does not look at the return value. The sampler never stopped and never found out. Four lines. No requirement, no review, no decision: a default, which is what the word means when nobody has written the sentence down.

Now the awkward part, because lesson 046 praised this chip and was right to.

046's claim is that the buffer discards on an acknowledgement rather than on a clock or a memory ceiling, and that this separates a store you can trust with the only copy from one you cannot. The first half survives the weekend without a scratch: the chip threw away nothing it had accepted. 046 even named the behaviour in advance, in a sentence about caches, that a cache answers full by throwing data away and a queue answers by saying no, and that the only eviction policy which will not lose a write is the one that refuses the write.

That sentence has a clause nobody has written. Refusing a write protects everything you already hold and destroys everything that arrives afterwards. A queue that says no has not prevented the loss. It has chosen who takes it, and lesson 041 already told you who: backpressure travels up the chain until it reaches something that cannot slow down, and the end of Galewatch's chain is the wind. The last link in a backpressure chain does not slow down and does not retry. It forgets.

One more correction, and this is the first lesson in a position to make it. Every figure this course held about that chip was a lower bound. Lesson 017's four hours and lesson 041's seven and a half are measurements of two outages, not of the chip, because an outage that ends before the buffer fills tells you about the weather. 16,384 is the first real capacity figure anybody has, and it belongs to one farm. Lesson 053 established that Tarrow Ridge runs the old generation of controller, that other farms do too, and that the list of which farm has which is an artefact nobody at Galewatch has maintained, so nobody can say which farms share that number. Lesson 041's seven and a half hours across Ardnave Point's three hundred turbines is the only figure anybody holds for any other farm, and it is a lower bound too, about the first of August.

A buffer makes you late, a writer makes you different

Lesson 024 put a buffer's value in one sentence. Queues, buffers and deadlines make you neither consistent nor available; they convert an outage into a delay, and a delay is the one failure a customer sometimes forgives.

That holds for exactly as long as nobody else can write the same thing. Lesson 034 had already sorted this and priced it. Costing a vector clock for Galewatch at one counter per writer and four bytes a counter, it refused the idea on better grounds than money: every reading is its own key, written once, never updated, so Galewatch has no two writers touching the same thing and would be paying to detect a conflict that cannot occur. 034 handed the paying case to today, two copies of one thing edited offline.

Say it as a rule, because it decides everything that follows. A device is a buffer while it is the only author of what it holds, and a writer the moment it is not.

Tarrow Ridge is pure first case. Lesson 019's unique constraint on (turbine_id, recorded_at) is the entire merge function, and it never has a conflict to settle, because the two copies of a reading it might compare are the same reading. Which is why every Galewatch outage in this course has produced a backlog or a loss and not one conflict. The weekend cost two million readings and there was nothing to merge, because that chip is a buffer with bad manners rather than a writer.

Lesson 050 sorted the other direction by what a conflict costs, which is a question about the data rather than about the deployment. Some things have no second writer. Some are a merge, a basket or a document or a set of tags, where last write wins will lose data quietly. And some are a lawsuit, a seat or a balance, where there is no merge because the conflict is the product being wrong. 050 named 058 as the lesson that owes the middle pile an argument.

The middle pile needs a shop.

Marlow takes a stall

Marlow Books is an online bookshop run by four people that exists only in this course, with 1.2 million titles and a founder who deploys on Friday afternoons. On Saturday 3 October 2026 it took a stall at a book fair in a hall with a concrete floor and no signal anywhere in it.

The founder had shipped stall mode the night before, which is exactly the kind of thing he does. One page in the shop's own admin app, orders kept in the browser's local storage, sync when you get a bar or two outside. Two phones at the stall, one his and one belonging to a colleague who was working the other end of the table.

The hot item was a numbered edition of a prize winner, eight copies in the warehouse and nothing at the stall but the sample. Both phones had synced at eight in the morning and both knew the number was eight. The founder took seven orders over the day. The colleague took two. At ten past six, in the car park, both phones found signal within a minute of each other and sent what they had.

Nine orders. Eight copies. Nothing anywhere in the system had done anything wrong.

Lesson 015 is why that last sentence is true. July's oversell, twelve signed first editions that sold thirteen, ended with the single statement the shop still runs:

UPDATE books SET stock = stock - 1
 WHERE isbn = '9780571358939' AND stock > 0;

Check the affected row count. One means the book is yours, zero means somebody got there first and you have not charged anything. The sync ran nine of those at ten past six. Eight changed a row, the ninth changed nothing, and the shop's stock count was correct at every instant of that day.

Somebody still had to email a customer who had walked out of the hall holding a receipt.

015 explained why its statement works, and that explanation is the whole of today: there is no gap for a second buyer to fit into, because there is no moment where the decision is made somewhere other than where the data lives. Offline-first is the deliberate choice to decide somewhere other than where the data lives. That is its definition rather than a side effect of it, and nothing will run that UPDATE from a hall with a concrete floor.

Offline-first does not break your invariant. It moves the moment you find out you broke your promise. The invariant has a database enforcing it. The promise has somebody with a phone.

The founder got the important half right by instinct: the phones synced their orders, not their view of the number. Payment waits until the shop ships, since a card sits in lesson 019's third pile and there is no version of charging one from a basement with no signal in it.

Reach for the other design and July comes back. Both phones read 8, one computes 1, the other computes 6, and each writes its answer where the stock lives. That is the lost update, the textbook bug lesson 015 said Marlow had already fixed by sending stock = stock - 1 rather than stock = 7. Whichever write lands second wins, so whichever phone's clock reads later wins, and lesson 034 measured what a clock is worth here: a median offset of four tenths of a second and a worst case of forty one, on machines with a job and a time daemon, which is a kinder case than two handsets that spent a day in a pocket. Last write wins is the lost update with a better name and a network partition for an excuse. Lesson 050 named the cost, that it loses data quietly, and quietly is the load-bearing word: the losing write leaves nothing behind, so the loss cannot appear on any graph somebody might think to build.

Which leaves the stock count safe and several other things on those phones not. A regular customer's delivery address, corrected on one phone at eleven and on the other at four. A gift message. A wish list. None of those has a guard, because none of them has an invariant, and for those the server has to merge.

Know that it is a conflict. Here is where lesson 034's vector clock earns the bytes it could not earn at the farm: one counter per writer, incremented on your own write, merged by taking the larger of each position. The founder's write carries {founder: 1, colleague: 0} and the colleague's carries {founder: 0, colleague: 1}, each with an entry larger than the other, so the server knows the two are concurrent rather than ordered. Not later. Not stale. Concurrent, and known to be, which is the one thing a timestamp will never tell you.

Make the merge total. Stop sending the value and send the operation: minus one, from this device, named. Decrements add up to the same total in any order, which is the property lesson 039 used on this shop when it called stock = stock + 1 commutative. A type whose merge is defined so that any order of merges gives the same answer is a conflict-free replicated data type, a CRDT, and a counter of that shape is the simplest one there is.

Then watch what it costs here, because this is where the textbook stops and the shop keeps going. A CRDT guarantees that every copy ends up agreeing. It has nothing to say about whether they agree on an answer you can ship. Replace AND stock > 0 with a commutative counter so that the merge can never fail, and nine decrements against eight copies converge, correctly and deterministically, on minus one, which is July's row to the digit. A commutative merge buys agreement by deleting the guard that was doing the work.

Lesson 023 settled the underlying point and it bears repeating here. July's thirteen buyers all read the true row, by primary key, on the primary, in 0.2 milliseconds, and the shop oversold anyway. The strongest read available was never the problem. The invariant was.

You cannot merge an invariant, so divide it

Stock is not a number. It is a promise that the count never goes below zero, and that promise is a statement about every writer at once. There is no merge function for it, because by the time two writes meet, both customers are holding a receipt.

Which leaves one move, and a design review usually walks straight past it. Divide the invariant before anybody goes offline.

Give each phone a reservation in the morning. Four copies to the founder, four to the colleague. Each device decrements against its own four and refuses at zero, with no server in the loop and nothing to merge, because the two phones are now the sole authors of two separate things. Which is today's rule run backwards: make each writer the only author of what it writes and there is nothing left to detect.

Price the day the shop actually had. The founder faced seven buyers and could serve four, so three people at the front of the table hear no. The colleague served two and went home with two copies still reserved and unsold. Six sold out of eight, and not one promise made that the shop could not keep.

Against the day that really happened, where the shop sold eight and disappointed one by email, the reservation sells six and disappoints three at the table. Two sales and two extra unhappy customers, to move a refusal from the car park at ten past six to a conversation at the stall in the afternoon.

That is the whole trade. A reservation turns a correctness problem into an inventory problem, and the inventory problem is one you can see. For a numbered edition, where the alternative is an apology and a refund, it is a bad deal. For a seat it is a magnificent one.

Stagefront, the ticketing service in this course where a stadium show goes on sale at exactly 10:00 and two hundred thousand people press the same button in the same minute, is in lesson 050's third pile, and it shows what else the division turns on. A seat divides perfectly, because 14F is either in your block or it is not, and 050 wrote that rule already: give every entity in that pile one home and route writes by entity. Marlow's eight copies are interchangeable, which is why dividing them strands two under a table. Dividing a fungible invariant strands inventory, and dividing an identified one costs nothing but the routing. What Stagefront cannot do is hand a block to a device and let it go offline all afternoon, because an unsold seat in a disconnected pocket is a seat nobody else can sell and a show that starts anyway.

So a reservation is the bound this course keeps finding, pointed at correctness instead of at load. Lesson 017 said the queue did not make Postgres faster, that the total work never moved and all that moved was who waits and where. The shop still cannot sell nine copies of eight; it just finds out at the table. Nobody can derive how big a reservation should be, for the same reason lesson 055 admitted nobody can derive a scale-in threshold. Divide by expected demand rather than evenly, hold back a remainder nobody can spend offline, and take the founder's sense of which of them sells more over any formula on this page.

What a device owes you on the way back

An offline writer owes you three things, and this course built all three for other reasons.

Name your own writes. A device that writes while disconnected has to say what each write is before anybody can deduplicate it, and lesson 019 said who does the naming: for anything a customer initiates, the only party who knows what one intent looks like is the client itself. An idempotency key minted on the phone is exactly that, and it makes the sync safe to replay as often as a car park demands.

Make the key a counter rather than a random string and a second property comes free. A random key makes a write nameable. A counter makes a missing write noticeable, because a gap in a sequence is a question and a missing random string is nothing at all. Run that back at Tarrow Ridge: with lesson 034's (turbine_id, seq), the ingest box would have seen the sequence jump by 33,116 at ten to ten on the Sunday, and the Monday would not have needed an engineer, an insurer or a query. The counter has to be stamped when the sample is taken rather than when it is stored, or the firmware that refused the write closes the gap behind it and you are back to counting rows.

Which is the third lesson to ask for that counter, and the first where the bill looks small. Lesson 034 wanted it so a reading's identity would not rest on a clock. Lesson 057 wanted it because an identifier with room between its values is one a forger can use, and priced it at new firmware on every turbine, which for the older ones means an engineer and a van. Today wants it so a drop is visible. One change, one van, three debts paid.

The van is lesson 053's and it has a date. Galewatch does not own these turbines, the owners do, and Tarrow Ridge is serviced in March and in September. September's round has been and gone, so anything needing a handset at the base of a tower there happens in March, alongside the last step of a key rotation 053 left in the same maintenance plan.

Delete only on an acknowledgement. Lesson 046 named this as the property that makes the hillside chip trustworthy, and it is the property a hand-built sync layer gets wrong first. A till that clears its local orders on a timer, or when the sync screen says done, or when the app is next opened, is a chip discarding on a clock. The founder's phone has to keep every order until the server has said which ones it holds. Not nine. Which nine.

Report what you dropped. The chip cannot do that today and the ingest boxes could do it this afternoon, with no firmware anywhere. Lesson 051 asked Galewatch to store counts rather than rates, a row carrying expected, received and available, so a low number can be audited instead of merely believed. Lessons 034 and 037 between them already put two clocks on every record, the time the turbine stamped and the time the ingest box received. Take the gap in received time for a turbine, divide by two seconds, compare it against the readings that actually arrived in that batch, and you have the drop count the firmware will not give you until March.

What 051's three integers cannot tell you is which policy threw the readings away. They say a reading is missing; the position of the hole says who dropped it. Together they separate a report with a gap in it from a report that is quietly wrong, which is lesson 052's finding aimed at missing rows rather than a missing column: an aggregate answers from the rows it has and says nothing about how many it skipped. Tarrow Ridge's Saturday holds data from midnight to 15:26, which is 64 percent of the day, and a monthly mean turns that into one number in a document an insurer is reading.

One promise does come free, and it is the only thing here that gets easier offline. Lesson 023 defined read-your-own-write as the guarantee that once a write of yours is accepted, every later read by you sees it, and a device reading its own store cannot fail it. 023 also named the version that breaks: across devices it is a different promise, about a human being, where the mechanism you bought was about a connection. Offline-first makes that one worse than it has ever been, because the device is the session and the session lasts for days. At ten past six both phones were Marlow Books, and neither could see the other's day.

Recap

The position of the hole names the policy. A fixed-size store at the edge can overwrite the oldest, refuse the new, stop producing, or fall over, and your own data narrows that to two of the four before anybody opens the source. Tarrow Ridge came back holding the beginning of the storm, so it refuses rather than overwrites.

Refusing a write chooses who takes the loss rather than preventing it, which is lesson 046's eviction sentence finished. The last link in a backpressure chain does not slow down and does not retry. It forgets, and lesson 041 named what the last link at Galewatch is.

Every figure this course had about that chip before today was a lower bound, because an outage that ends before the buffer fills is a measurement of the weather. 16,384 is the first that is not.

A device is a buffer while it is the only author of what it holds, and a writer the moment it is not. Lesson 024's delay survives the first case and dies in the second, where an outage becomes a divergence. Lesson 034 had already shown Galewatch to be the first case: every reading is its own key, written once, never updated, so the farm lost two million readings over one weekend and produced no conflict at all.

Offline-first is the deliberate choice to decide somewhere other than where the data lives, which is the exact property lesson 015's conditional update depends on. It does not break your invariant. It moves the moment you find out you broke your promise, and the shop's stock count was right all day while a customer went home with a receipt for nothing.

Last write wins is the lost update with a better name, settled by a clock lesson 034 has already discredited, and it discards a write without leaving a trace. A vector clock tells you the two writes were concurrent instead of guessing, for eight bytes at two writers and far too much at one per turbine. A commutative merge goes further and buys agreement by deleting the guard that was doing the work: nine decrements against eight copies converge beautifully on minus one, which is July's row to the digit.

You cannot merge an invariant, so divide it in advance. A reservation turns a correctness problem into an inventory problem you can see, and at the stall it cost two sales and two more disappointed customers to move a refusal from an email to a conversation. A fungible invariant strands inventory when you divide it and an identified one does not, which is why a seat routes to a home and eight interchangeable books sit under a table.

An offline writer owes you a name it minted itself, a deletion that waits for an acknowledgement, and a count of what it threw away. Make the name a counter and a missing write becomes noticeable rather than invisible, which is the third lesson to ask for the same four bytes and the first to find them cheap.

Check your understanding

1. Tarrow Ridge's firmware refuses the new reading when its log is full, and nobody has ever filled the log at Ardnave Point, the three hundred turbine site. Say what you would ship this week, with no firmware change anywhere, that would tell you which of the four behaviours Ardnave Point's controller implements, and what would have to be true of the weather before it answered.

2. A colleague proposes changing the old firmware to overwrite the oldest reading instead, on the grounds that a dashboard nine hours behind is useless. Give the strongest version of their argument, name the customer it is wrong for, and say what you would do about the fact that both policies get chosen before anybody knows which outage they will meet.

3. Marlow's founder wants stall mode for next year's fair and, having read this lesson, asks for reservations and a conflict-free counter. Say which of the two changes what a customer experiences, what the other is still worth having, and what you would put on the phone's screen when a reservation hits zero with somebody standing there.

4. Stock is not the only thing two offline phones can touch. Take a customer's delivery address, a wish list and a gift message, sort them into lesson 050's three piles, and say for each one whether a last write wins merge is defensible and how you would ever find out that it had lost something.

5. You inherit a sync layer that clears a device's local queue when the server returns 200 to a batch of writes. Say what goes wrong on a partial failure inside that batch, what you would change about the response, and why lesson 019's three piles decide how hard the fix is.

6. Galewatch wants to sell the insurer a weekend report that is honest about the hole. Using what lesson 051 asked for and what today adds, say which numbers appear on it, and what you would print for the stretch where the only true statement is that the turbine was measuring and nobody kept the result.

7. Take a system you work on with any kind of client-side store, a browser tab included. Say whether that store is the only author of anything it holds, what its policy is when it fills up, and how you would answer the second question without reading the code.

Next lesson

059 Audit Logs and Immutable Records. Today's drop count, reservation and sync receipt are all claims about what happened while nobody was watching, and lesson 057 closed on the observation that a receipt your own incident can delete is not one, so 059 is about the record you cannot quietly edit afterwards.