A controlled layer between enterprise applications and multiple model providers, so access, routing and policy are decided in one place instead of separately in every application.
Enterprises rarely settle on a single model provider. Teams adopt whichever model suits their problem, and each integration arrives with its own SDK, credential, error semantics and streaming format.
Without a common layer, provider choice becomes an application-level decision replicated dozens of times. Credentials spread across services, usage becomes impossible to attribute, and changing a provider means changing every caller. Governance turns into a review meeting instead of something the system enforces.
Shared ownership stated as such. Nothing here claims sole authorship of a platform.
Treat the provider as an implementation detail. Callers address a logical model; the gateway resolves that to a concrete provider, credential and endpoint at request time, which is what turns routing and failover into a configuration change instead of a code change.
Express governance as composable policies over the request lifecycle, not as branching logic buried inside the proxy. Each policy has a narrow contract, and the order in which they run is itself configuration.
Keep the streaming path first-class. Accounting and termination must work on a response that is still being produced, so usage is recorded from what was actually delivered, not from what was asked for.
The ones with a real cost on the other side.
A provider abstraction is only worth having if it still tells operators what they need to know: which model actually served the request, which provider failed, what the call cost. Hiding all that makes the layer easier to describe and a lot harder to run.
Access decisions must precede anything expensive, and accounting must follow the response. Encoding that ordering explicitly keeps the guarantee inspectable instead of implicit in the call sequence.
A governed platform layer through which applications and agents reach language models without each one managing its own provider integrations, with access control, routing and usage accounting applied in one place.