LLM gateway vs OpenRouter is a decision about where the routing layer lives, not a model contest. An LLM gateway is a proxy between your apps and model providers: one API for many models, plus routing, retries, fallbacks, caching, rate limits, key management and cost tracking. You can run one yourself (the open source default most teams mean is LiteLLM) or use a hosted one. OpenRouter is the best known hosted option: one OpenAI compatible base URL, one credit balance (or your own keys), automatic failover and no proxy to operate. Pick OpenRouter when you want breadth today with almost no infrastructure. Pick a self hosted gateway when data residency, org level keys or high spend make the managed fee the wrong trade.
They are not mutually exclusive. LiteLLM can call OpenRouter as an upstream provider, so you keep local policy while OpenRouter handles catalog breadth. This guide explains the category first, then compares the two. Fees, features and release details were rechecked against official pages on October 24, 2026. For where a gateway sits in an agent stack, see what is an agent harness.
LLM gateway vs OpenRouter at a glance
| Item | Self hosted LLM gateway (LiteLLM) | OpenRouter |
|---|---|---|
| Shape | Proxy and SDK you deploy | Managed multimodel API |
| Who runs it | You (Docker or Kubernetes, usually with Postgres) | OpenRouter |
| Provider keys | You hold them | OpenRouter's, or your own (BYOK) |
| Billing | Each provider directly, plus your infrastructure | Credits with a platform fee, or BYOK |
| OpenAI compatible | Yes | Yes (https://openrouter.ai/api/v1) |
| Team keys and budgets | Virtual keys, budgets, model allowlists | Budgets and spend controls on paid plans |
| Local or VPC models | Yes (for example Ollama, Bedrock, Azure) | Only the providers OpenRouter lists |
| Best fit | Platform teams, compliance, high spend | Fast start, product teams, trying many models |
What is an LLM gateway?
An LLM gateway is a single front door for model traffic. Your apps and coding agents send requests in one format, usually the OpenAI chat completions shape, to one base URL with one kind of key. The gateway decides which provider and model serve each request, translates the call, and records what it cost. Switching from one provider to another becomes a config change instead of a code change.
What a gateway does
- Routing: send each request to a model by alias, weight, latency, cost or a custom rule.
- Retries and fallbacks: when a provider errors or rate limits, retry or move to the next model or provider.
- Caching: return a stored answer for an identical request, so you skip the provider call.
- Rate limits and budgets: cap requests and spend per team, app or key.
- Key management: apps get gateway keys, and only the gateway holds the real provider keys.
- Logging and cost tracking: one place to see tokens, spend and errors across every provider.
Self hosted vs hosted gateways
Self hosted gateways run on your infrastructure, so prompts and keys stay on your network. LiteLLM (MIT licensed, apart from an enterprise/ folder), the Portkey gateway (MIT) and agentgateway (Apache 2.0, a project of the Linux Foundation's Agentic AI Foundation) are examples. You own uptime, upgrades and patching.
Hosted gateways are someone else's proxy. OpenRouter, Vercel's AI Gateway and Cloudflare AI Gateway all fit here. They differ mostly in billing and extras. OpenRouter and Vercel sell model access through prepaid credits. Cloudflare AI Gateway can bill through Cloudflare or sit in front of your own provider keys, and adds caching, rate limiting and analytics.
Gateway vs router vs SDK
The words overlap, so here is how we use them. A gateway is a network service your apps call. A router is the logic inside it that picks the model; OpenRouter is a hosted gateway whose router is its main feature. An SDK such as the Vercel AI SDK is a library inside your app that gives one API across providers but adds no shared keys, budgets or logs across apps. Many teams use an SDK in the app and point it at a gateway. Our Vercel AI SDK guide shows that setup.
LLM gateway vs API gateway
A plain API gateway (Kong, AWS API Gateway, nginx) routes any HTTP service and handles auth and rate limits. An LLM gateway understands model traffic: token cost, model aliases, provider failover and prompt logging. Companies often run both, with the API gateway at the edge and the LLM gateway behind it. An MCP gateway is different again: it fronts tools, not models (see how to build an MCP server in Python and MCP vs A2A). LiteLLM's docs now cover MCP and agent gateway features in the same proxy, which is convenient, but tool design still needs its own care.
What OpenRouter is
OpenRouter is a hosted unified API over hundreds of models and many providers. Per its quickstart, you call https://openrouter.ai/api/v1/chat/completions (or point the OpenAI SDK at that base URL), pick a model slug, and OpenRouter routes, falls back and bills. Its pricing page lists 500+ models from 80+ providers on paid plans, and 25+ free models for trials.
OpenRouter's own OpenRouter vs LiteLLM post frames the choice as where the routing layer should run: their infrastructure, or your Docker, Postgres and Redis stack. That framing is fair. Coding agents such as Cline and Aider accept an OpenRouter key, so one config gives access to many models.
Fees and cost math
Figures below match the official pages on October 24, 2026. Prices change, so check before you buy.
OpenRouter
From the FAQ and pricing page:
- Model prices are passed through from providers with no markup on inference.
- OpenRouter charges a platform fee when you buy credits: 5.5% on Standard pay as you go and 8% on Business, with discounts on Enterprise.
- BYOK (your own provider keys): pay as you go plans include $25,000 a month of list price inference with no BYOK fee; above that the fee is 5% of what the same usage would cost on OpenRouter, deducted from credits (BYOK docs).
- The Free plan is limited to 50 requests a day and free models: fine for trials, not production.
LiteLLM
From LiteLLM pricing and the docs:
- The open source gateway is free to self host. You pay providers directly, plus your own compute, Postgres and optional Redis.
- Enterprise is a license priced to your gateway's scale, never per token, and adds governance, security features and support on the same gateway.
A simple way to compare: multiply your monthly model spend by the OpenRouter fee and set that against the full cost of running a proxy, including the engineering time to keep it patched. At low spend the fee is usually cheaper. At high spend, or when compliance requires your own network anyway, self hosting wins.
Routing, keys and control
OpenRouter optimizes for defaults: automatic provider fallback when an upstream errors, auto routing, and filters for price, latency, throughput and data policy. Budgets and spend controls are on paid plans, and SSO is an Enterprise feature.
LiteLLM optimizes for policy you own: virtual keys per team or app with budgets and model allowlists, spend tracking, fallbacks, load balancing, guardrails and an admin UI (proxy overview). You can point one gateway at Azure, Bedrock, Groq, Ollama and OpenRouter in a single model_list.
Data path: with LiteLLM, prompts hit your proxy first, then the provider. With OpenRouter, prompts pass through OpenRouter's layer first; read its data policy and zero data retention options for your threat model. A rule that traffic may only leave through systems you approve usually points to self hosting.
Security: a gateway holds every key
Self hosting moves the key store onto your infrastructure, which makes the gateway a prime target. On October 21, 2026, LiteLLM 1.107.0 was published to PyPI and as Docker images with a credential stealer that, among other things, dumps the provider keys stored in a LiteLLM proxy database, per the public security report. It was the project's second compromised PyPI release of 2026, after 1.82.7 and 1.82.8 in March. Our LiteLLM compromised guide has the checks and the rotation list. On October 24 the maintainers resumed releases with v1.106.3, and the release notes say images and Helm charts are now signed with a new cosign key, so verify with the new key.
Whichever gateway you self host, pin exact versions with hashes, pin images by digest instead of floating tags, and give the proxy only the secrets it needs. With a hosted gateway, patching is the vendor's job, though your gateway key and any BYOK keys still need the same care.
Using both together
A common production pattern:
- Deploy LiteLLM inside your network for virtual keys, budgets and logging.
- Add OpenRouter as one upstream for models you do not want to contract directly.
- Keep direct Azure, Bedrock or Ollama routes for regulated or local traffic.
Because both speak the OpenAI chat completions shape, switching a client is mostly the base URL and the key.
When you do not need a gateway
If you call one provider from one app, a gateway is extra moving parts. The provider's SDK, or a library like the Vercel AI SDK, already gives you retries, and the provider's console shows spend. Add a gateway when a second provider, a second team or a budget rule appears, or when you want to stop handing raw provider keys to every app and coding agent.
Decision guide
Choose OpenRouter if you: need many models this week without collecting provider keys, have no platform team to run Postgres, Redis and upgrades, or are prototyping and want easy model shopping.
Choose a self hosted LLM gateway (LiteLLM) if you: must keep the routing layer inside your network and can own its patching, need virtual keys and hard budgets per team, already pay providers directly at scale, or want one internal endpoint for apps, coding agents and batch jobs.
Choose both if you want org policy locally and marketplace breadth upstream.
FAQ
Is OpenRouter an LLM gateway?
Yes, a hosted one: a managed OpenAI compatible front door over many providers with routing and failover. People still contrast "LLM gateway" with OpenRouter when they mean self hosted gateway software such as LiteLLM.
Is LiteLLM free?
The open source gateway and SDK are free to self host, including commercially, under the MIT license that covers everything outside the repository's enterprise/ folder. Enterprise features and support are sold separately, priced to scale rather than per token.
Does a gateway add latency?
Yes, one extra network hop plus the gateway's own processing on each request. Run a self hosted gateway close to your apps, and measure time to first token with and without it. A gateway that fails over quickly or serves cached answers can still lower the latency users see.
Can a gateway cache responses?
Many can. LiteLLM supports Redis, Valkey, S3 and semantic caching; OpenRouter's response caching returns identical requests from cache with no billing; Cloudflare AI Gateway caches identical requests too. Caching only helps for repeated identical prompts, so it suits tests and fixed lookups more than open chat.
Do I need a gateway for one provider?
Usually not. One app and one provider work fine with the provider's SDK. A gateway starts paying off with a second provider, shared budgets across teams, or when you want apps to hold gateway keys instead of raw provider keys.
Can I use LiteLLM and OpenRouter together?
Yes. Configure OpenRouter as a provider in LiteLLM's model_list. You keep local keys, budgets and logging, and OpenRouter supplies its catalog and failover.
