Rate limiting and budget enforcement for LLM traffic across users, agents, organisations, models, providers, tokens and monetary cost.
LLM consumption is not well described by request counts. The same number of calls can differ by orders of magnitude in tokens and in cost, and the budget anyone actually cares about is measured in money, not in call counts.
Organisations need to cap consumption along several dimensions at once: per user, per agent, per application, per model, per provider. Worse, some of the quantities they want to cap are only known partway through serving the request. Token counts are not final until a response completes, and cost depends on provider-specific pricing for input and output separately.
Shared ownership stated as such. Nothing here claims sole authorship of a platform.
Separate admission from accounting. A request is admitted against what is known up front and reconciled against what actually happened, because waiting for certainty before admitting would put provider latency inside the rate-limit decision.
Let the quota key be derived from the request context instead of pinning it to one field, so the same policy expresses per-user, per-agent, per-application or per-key limits without a separate implementation for each.
Give virtual credentials an identity of their own. Once a key is the unit of access, attribution, restriction and central revocation all follow from the same object instead of requiring three mechanisms.
The ones with a real cost on the other side.
Request and token ceilings are proxies for the thing organisations actually care about. Treating monetary cost as its own dimension means carrying provider pricing in the evaluation path. That is more moving parts, but it caps what the budget owner is actually exposed to.
Admitting on an estimate and settling afterwards means brief over-admission is possible under concurrency. The alternative, blocking until the totals are certain, would add a provider round-trip to every governed call. Bounded imprecision was the better trade.
Centralised governance that lets organisations attribute, limit and control AI consumption across identities, models and providers, with monetary budgets as a directly enforceable dimension.