Verified 2026-11-18 · rates follow the live catalog

LLM gateway: one API in front of many models

An LLM gateway sits between your application and the companies that run models. You send every request to one address with one credential, and the gateway decides where it goes, reshapes it if the destination speaks a different dialect, and tallies the cost. Whether that is worth a network hop depends on how many models you use and how much you care when one of them has a bad hour.

What a gateway is for

Questions to ask before you choose one

Which request formats does it accept?

Openference serves OpenAI chat completions, the OpenAI Responses API and the Anthropic Messages API on the same host and the same key. When the destination speaks the format you sent, the body passes through unchanged. When it does not, the request is converted, including tool calls and streaming events, and the reply is converted back. See what is and is not translated.

What happens when an upstream fails?

For each model there are multiple upstream routes, across different providers and keys. Selection is weighted and uses cooldowns. A route that returns repeated errors is temporarily taken out of rotation, and healthy routes are preferred. Upstream failures are never charged to your quota.

Does it publish what things cost?

Every one of the 53 text models in the catalog has a public per-million-token rate, with cached-input prices and off-peak schedules where they exist. The cost-per-request guide works those rates into what a single call costs, and OpenRouter alternatives compares what other gateways charge for the same models and on top of them.

Can you scope a credential?

A key can be limited to a subset of models. A request for a model outside that set is refused with a 403, and the model list that key sees contains only what it may call.

Does streaming survive?

Server-sent events are forwarded as they arrive. When a stream has to be converted between formats, the client still gets a well-formed terminal event even if the upstream stream ends abruptly.

Gateway or direct provider keys

When each approach fits
SituationDirect to the providerThrough a gateway
One provider, one modelSimpler, one fewer hop, every provider-specific feature availableLittle to gain
Comparing or mixing several modelsA new account, key, bill and client per providerOne key, one base URL, one bill
A tool built for a different API formatNeeds an adapter you write and maintainTranslation is part of the service
Provider-only featuresAvailable as the provider ships themOnly what the gateway passes through

A gateway cannot make a model better than it is, and it adds latency of its own. If the table's first row describes you, call the provider directly.

Supported tools

Cursor, Cline, Codex CLI, Claude Code, OpenCode, Kilo CLI, Pi, Open WebUI and SillyTavern are listed as supported in the compatibility matrix, with a smoke test for each. Setup guides: Cursor, Claude Code, Codex CLI, OpenCode, Pi, Open WebUI, Cline.

If you already point a tool at an OpenAI-style base URL, the change is one setting. The LLM proxy page walks through exactly what changes in a request.

Model catalog · Pricing · Comparisons · Cost guides · Integration guides · FAQ
Guides: GLM coding plan alternative · Free AI API · LLM gateway · LLM proxy · Cheap AI API · OpenAI-compatible API
Model APIs: GLM API · DeepSeek API · Kimi API · Qwen API · MiniMax API