Authentication and Authorization at the Edge

Marlow Books is an online bookshop run by four people, and it exists only in this course. Its admin page is older than most of its bugs: one form for changing a price, one button for the overnight import, and, since September, the two fields that decide whether the shop is taking orders at all.

For years that page was protected by one line in the handler. If the signed in customer's id was not 1, return a 404. The founder is customer 1. Nobody else has ever been.

In November the two buyers needed it.

They are the two people lesson 010 built the publisher page for, and the courier had started missing collections, and the founder was tired of being the only person who could put a sentence on the shop. So the line came out of the handler and went somewhere that felt tidier: one rule at the managed balancer lesson 006 installed in January. Any request whose path starts with /admin must carry a session cookie. If it does not, bounce it to the sign in page.

Twenty minutes. Tested by signing out and watching the bounce, then signing back in and watching the form load. Both buyers could get in. The founder made a note to add a proper staff check later, in the way one does.

What the balancer can see is a path, a method and the headers. What it cannot see is lesson 007's sessions table, which lives in Postgres on box A and holds a customer id. So the question the handler used to ask, is this customer the founder, became the question the balancer could ask, is there a cookie here. On the Christmas peak lesson 007 counted eight thousand people signed in at once, and from that afternoon in November every one of them could open the page where the prices are.

Nobody did. What happened instead was worse in a quieter way.

On the second Saturday in December the courier did not come. The packing room had a pile and one of the two temporary staff lesson 009 counted at the Christmas rush had been sent the admin link back in November, by a buyer, so somebody other than the founder could put the sentence up. They opened it on their own phone, signed in to their own customer account, and it worked, exactly as it had in November, because nothing had changed and nothing was ever going to.

The form has three buttons, because lesson 028 widened that field in October from a boolean to "open", "closed" and "delayed". They typed a sentence about the courier and pressed Closed, reading it the way anybody in a warehouse reads it. The courier is closed today.

10:40. The write landed on whichever box the balancer's least connections rule picked, and lesson 027 put those two fields in a file on each box rather than in a database, deliberately, so they survive the mode where the database is the problem. Lesson 027 also named the price of that: two files can disagree. So one box now said the shop was shut and the other said it was open, and the small JSON request that lesson 022 invented and every book page fetches on load went to whichever box was free.

Which means the shop was not half closed. Every customer's next page load was a coin toss.

Nobody emailed. The sentence was about a courier, in plain words, which is precisely what lesson 027 said a degraded mode must do, so the shop looked like it was having a considered day rather than a broken one. The best available banner bought forty five minutes of silence.

At 11:25 a buyer messaged the founder to ask why the site was announcing a collection problem when the courier was booked for Monday anyway. Fixed by 11:29.

Price it with the shop's own numbers and the assumption named. Lesson 027 derived one order every fifty seconds from an ordinary forty requests a second, carrying lesson 009's Christmas conversion rate across, which nobody has checked. Forty nine minutes is 2,940 seconds, so about fifty nine orders' worth of buying time, and a December Saturday is busier than an ordinary Tuesday, so treat fifty nine as a floor. Half of those page loads came back with a grey button. An unknown number of those people reloaded and bought anyway, which is the only time in this course that the retry layer lesson 020 found installed on every client has done anybody a favour.

The fix took an afternoon and it went back into the handler: a staff column on the customer row, read on every admin request, checked against the id in the session. The balancer rule stayed, and the reason it stayed is the whole of today.

Two questions that are not the same question

Authentication is the question of who you are, and whether you can prove it. Authorization is the question of whether you may do this particular thing to this particular object. Two words a letter apart, doing jobs that do not resemble each other.

Authentication has one answer per caller. Marlow's temp is customer 3,114 and stays customer 3,114 whatever page they open, so the answer can be computed once, early, by anybody, and carried around. That is why it moves outward so easily.

Authorization has one answer per caller per verb per object. May customer 3,114 read order 4,471 depends on who placed order 4,471. May they change the price of a book depends on a column in a row somewhere. May they cancel an order depends on whether it has already shipped. There is no answer to carry, because there is no question until the object is known.

So the structural fact, and it is the reason the November rule felt fine:

The edge knows who is calling before it knows what they are calling about.

Push a check outward and you buy cheapness and lose precision, in that exact proportion. The balancer got a rule it could actually enforce and the question shrank to fit the place it was being asked. Nobody decided to stop checking for staff. The check was moved, and the part of it the new location could not express quietly fell off on the way.

You will find this in code as the pattern where every route is guarded and nothing is. A middleware that requires a signed in user, applied to every path under /account, is doing real work: it keeps strangers out. It says nothing at all about whether this signed in user may read /account/orders/4471, and that check has to sit in the handler, next to the row, because that is the only place the owner of the row is known.

What a token can prove

Lesson 007 built the vocabulary. A signed token is data plus a signature computed with a secret only the server knows, so the holder can read it and cannot edit it, and 007's verdict was that it is a cached authorization decision where the expiry is how long you are willing to be wrong. Today sharpens that phrase, because a token's subject is an authentication and nothing else in it is. Marlow measured the lookup it would save at about one percent of what its database already does and declined, which is still the right call for a bookshop.

Two things 007 left on the table, and both of them are today's.

A token is a bearer instrument. Whoever holds it is you, the way whoever holds a cinema ticket sits in that seat. No part of the mechanism asks how the holder got it. That makes every place a token can be copied from part of your security: a log line that records headers, a browser extension, an error report with a request dump attached, the screenshot somebody pastes into a support chat. I have seen a whole authentication redesign caused by an access log.

The second is what belongs inside one. Three things can be put in a token honestly: who the subject is, when the token stops being valid, and who the token was minted for. That last one has a name worth having, the audience, and it exists because a token your mobile app gets is a token your admin API should refuse. Without it, any service that trusts your signing key trusts every token in the company.

Everything else you put in is a decision frozen at signing time, and it is frozen for the whole lifetime you chose.

"role": "staff" in a fifteen minute token means a person you demoted at 09:00 is staff until 09:15. Lesson 023 named the shape and it is sharper than the staleness this course has been handling: a stale price shows a wrong number on a screen where a customer can see it and email you, and that is how Marlow found its May promotion bug in eighteen minutes. A stale permission lets somebody do something they are no longer allowed to do, and there is no screen anywhere in the world on which that renders. It fails silently, and it fails in the direction of yes.

Which is why "you cannot revoke a signed token" is repeated everywhere and is not quite true. You cannot un-sign it. You can keep a list of the ones you want refused, and the useful part is how small that list is, because a revocation list only has to hold what is both revoked and unexpired.

Marlow bans an account roughly never. Say once a week, with a fifteen minute token. Fifteen minutes out of the 10,080 in a week is 0.15 percent, so the list is empty almost always and holds one entry for a quarter of an hour about once a week. Stagefront, the ticketing service in this course where a stadium show goes on sale at exactly 10:00 and two hundred thousand people press the same button in the same minute, has the busy version: kick two hundred bot accounts a minute during an on sale, with a five minute token, and the list holds about a thousand entries. At sixty four bytes a token id that is sixty four kilobytes, which fits in every process on every box with room to spare.

Size is never what stops people. The list is a piece of fast changing shared state that every checking process needs, and lesson 048 owns that problem properly. Say the real objection out loud, though, because "signed tokens cannot be revoked" is the sentence that stops the conversation before anybody prices the list.

Galewatch, which collects a reading every two seconds from each of nine hundred wind turbines, has the version with no human in it. There is no password behind a turbine. What it carries is an API key, which is a password you posted to somebody, with no expiry, and no way to find out how many copies exist. Lesson 021 asked the reader to consider a customer running three hundred turbines on their own firmware, and left it open; as a credentials question the answer is bleak, because a key in flash on a machine bolted to a tower cannot be rotated without an engineer and a van. Lesson 053 owns rotation. The move that works when you cannot keep a secret is to shrink what the secret can do: that key may write readings for those turbine ids and nothing else, so a leak costs you fake readings from one farm rather than the fleet.

And the thing 028 could not do without any of this. Galewatch deleted an endpoint because a graph said five hundred and forty nine requests a day, and could not say that they came from one spreadsheet at one customer. An unauthenticated endpoint has no callers, only traffic. A key is how a graph learns a name.

Where to put the check

   edge            balancer          handler           the row
   sees a path     sees a path,      knows the         knows who owns
   and whether     a method and      customer id       it, and what
   a cookie is     a signed token                      state it is in
   present

   cheapest, weakest  --------------->  costliest, and the only
   answer                               complete answer

Read that as a sentence, since the spoken version skips tables and diagrams. An edge can tell a signed out stranger from a signed in one. A balancer or gateway, meaning whatever box terminates connections in front of your handlers, can verify a signature and an expiry without asking anybody, which is more than it sounds. Only the handler, sitting next to the row, can answer whether this person may touch this thing.

So the outer checks are worth having and they are worth being honest about. A check at the edge is a filter, not a decision. It sheds load and it shrinks blast radius, which are the two things lessons 021 and 027 spent themselves on, and Marlow's balancer rule does both: after the staff column went in, a signed out stranger still never reaches the admin handler at all. What it must never do is make the handler's check feel redundant, because the day the handler's check is redundant is the day somebody moves the route.

Two traps live out here.

The first is that a path prefix rule is a promise about a naming convention, and naming conventions are maintained by people in a hurry. Marlow's admin form posts to /admin/switch, so the rule covered the action as well as the page, and that was luck rather than design: the page and its handler were in one file. Had the switch been split into its own small endpoint, the way lesson 022 split the basket count out, and had it landed outside the prefix, the rule would have matched the door and not the room, and the temp would not have needed an account at all. Any stranger could have closed the shop. Run your own prefix against your own route table before you trust it, and remember that the route table grows on Fridays.

The second is caching, which is where the two halves of this course meet. Lesson 022's rule was that a cache key must contain everything the response depends on. A response that depends on who you are therefore needs a key containing who you are, and a key containing who you are is a cache with one user in it. That is not a tuning problem to solve. It is arithmetic. Lesson 023 put the general form plainly: no session promise survives an edge, because there is nobody there to carry a token or be sticky to.

Marlow already shipped the only answer that works, twice, without either time calling it security. Lesson 009 split the book page so the shell could be cached and the price read live; lesson 022 did the same split one radius further out after Anand's name and his three basket items were served to the twenty odd strangers in his city. The document at the edge belongs to nobody, and everything about a person arrives in a second small request that no cache touches. The edge's job with an identity is to not need one.

When the check is the thing that breaks

Put an identity service on the purchase path and you have added a dependency to the part of the system that makes money.

Lesson 003 priced Stagefront's purchase path exactly: a CDN at 99.99 percent, a balancer at 99.99, an application tier at 99.95, a ticket database at 99.95 and a payment provider at 99.9, multiplying to 99.78 percent and 19.3 hours a year. Add an identity service as good as their own application tier, 99.95 percent, and the chain becomes 99.73 percent and 23.6 hours. Four hours and change, bought with one hop.

Now the part that matters more, because Stagefront's real target is the per window one: at least 99.95 percent of purchase requests succeeding in under two seconds between 09:55 and 10:15, scored pass or fail on each of forty on sales a year. A 99.95 percent service is down 263 minutes a year. Forty windows of twenty minutes is 800 scored minutes out of 525,600, so spread those failure minutes evenly and the expected damage is 0.4 minutes a year. Twenty four seconds. You would sign that.

Do not sign it. Lesson 027 established that a shared dependency's failure minutes are not spread evenly, and an identity service is most likely to fall over at 10:00 on an on sale for the same reason everything else is: that is when two hundred thousand people sign in at once. The uniform assumption is the one doing all the work in that calculation, and it is false in the direction that costs you the window.

Which leaves the question nobody asks until the first outage. When the identity service times out, what does the gateway do?

Fail closed and one service's bad minute signs everybody out during your on sale. Fail open and you have built a switch that turns authorization off whenever one service gets slow, and lesson 020 established that a struggling dependency is the one your own clients hit hardest, so the slowness feeds itself once it starts. Lesson 027's test does not resolve this the way it resolves a banner: fail open on a suggestion, fail closed on a decision, and a check is never a suggestion.

The honest way out is to stop making the call. A signature verified inside your own process cannot be unavailable, because there is nothing to be unavailable. Lesson 007 priced signed tokens in milliseconds of database time and found one percent, which is the right answer to the question 007 was asking. The bigger reason to hold one is that it removes a dependency from the path, and an availability argument is invisible in a latency measurement.

The bill arrives as revocation delay, and on an on sale you can put a number on it. A five minute token means a scalper you kick at 10:03 keeps buying until 10:08, and lesson 003 said each on sale has about ten minutes of real intensity. Half your event. That is what 007's "how wrong you are willing to be" costs when you make it concrete, and it is also the strongest argument for paying lesson 048's price and pushing that little list of revoked ids to the twenty boxes lesson 006 counted.

The paths where nobody is signed in yet

Sign in, password reset, sign up, guest checkout. Lesson 021 named these as the paths with nothing to count and Marlow tightened per address limits on the first two in February, so the limiter half is built.

What makes sign in different from every other endpoint is that it is the one place a caller is trying to become somebody. Everywhere else a wrong answer leaks data. Here a wrong answer manufactures an identity, and then every later check in the system is working correctly and answering for the wrong person.

Credential stuffing is the attack that exploits it: take a list of email and password pairs leaked from some other site, try them all here, keep the ones that work. It needs no cleverness because people reuse passwords, and the list costs nothing. Take a success rate of one in a thousand, a number you should measure on your own logs rather than borrow from mine, and a list of a hundred thousand pairs is a hundred accounts.

Lesson 021's bind reads differently from this side. A limiter per address is a limiter per office, and the attacker's whole advantage is that they have more addresses than you have offices. Per address limits still belong on these paths and they are not the defence.

Two cheaper things are. The first is refusing to say which half was wrong: "no such account" turns your sign in form into a tool for testing whether an email address shops here, and there is no way to close that completely, because your sign up form has to refuse duplicates. That is an oracle, a way to make your system answer a question nobody asked it, and knowing where yours is beats pretending you have none.

The second is timing, and it is the one that gets missed. A password hash is deliberately expensive, call it a hundred milliseconds, because that expense is the only thing standing between a stolen database and every password in it. A lookup that finds no account does no hashing and comes back in something like the 0.2 milliseconds lesson 007 measured for a primary key read. That gap is the same oracle, machine readable, and nothing in your logs will ever show it. Hash a dummy anyway and return at the same speed.

Lesson 021 set thresholds by the cost of a false refusal. On this path there is a second cost to weigh against it, and they are wildly asymmetric: a false refusal is one annoyed customer retyping a password, and a false acceptance is every order that account ever places. Sign in is the one endpoint where you can afford to be slow, strict and slightly irritating.

Recap

Authentication asks who you are, authorization asks whether you may do this to this object, and only the second one needs to know what you are touching. That is why they separate so badly in practice: an identity can be computed once and carried anywhere, and a permission cannot exist until the row is in hand.

The edge knows who is calling before it knows what they are calling about, so every step outward buys cheapness and sells precision. Marlow moved one line out of a handler and into a balancer, and the question quietly shrank from "is this the founder" to "is there a cookie", which is how eight thousand signed in customers came to hold a key to the price form.

A check at the edge is a filter, not a decision. Keep the outer ones for what they are good at, which is keeping strangers off the path and load off the box, and never let their existence make the handler's check look redundant. Check the prefix against the route table, because the route table grows on Fridays.

A token is a bearer instrument, and every claim in it is a decision frozen for the length of its expiry. Subject, expiry and audience are honest contents. A role is a cached yes, and lesson 023's warning applies at full strength: a stale permission renders on no screen anywhere. A revocation list is smaller than everybody assumes, because it only holds what is both revoked and unexpired.

The best reason to verify locally is availability, not speed. Lesson 007 measured Marlow's saving at one percent of its database load and was right to refuse; on Stagefront's purchase path the same token removes a dependency worth four hours a year on paper and rather more in the minute that counts. What you pay is revocation delay, which on a ten minute on sale is half the event.

On the paths where nobody is signed in yet, the cost of a false acceptance is every order that account ever places. That is the one endpoint where slow, strict and irritating is the correct design, and where the leak is usually the timing rather than the words.

Check your understanding

1. Marlow's staff column is a boolean. Lesson 028 has already told you what happens next. Name the third case that arrives first at a four person bookshop, say what breaks when it does, and give the shape you would have used instead without adding a permissions system nobody asked for.

2. The balancer rule stayed after the fix. Write the argument for deleting it, then the argument for keeping it, and say which you would do at Marlow and which at Stagefront on an on sale morning.

3. Marlow wants to cache the small JSON request that carries the price, the checkout state and the customer's basket count. Using lesson 022's key rule, say what the key would have to contain, what hit rate you would expect, and what you would split out to get a cache that works.

4. Galewatch is asked by a customer to let their own dashboard call the readings API with one API key for all three hundred of their turbines. Say what that key can do if it leaks, what you would scope it to instead, and how you would find out today which of their turbines is actually using it.

5. Your gateway verifies tokens by calling an identity service with a 500 millisecond timeout. Write down what happens on a timeout today, whether that is fail open or fail closed, who decided it, and what the answer would need to be for you to be comfortable saying it out loud in a review.

Next lesson

030 Phase 1 Capstone: A Simple Web Service, End to End. Twenty nine lessons have each fixed one thing on systems that already existed, and the capstone builds one from nothing, with every decision in it citing the lesson that paid for it.