# LLM Gateway

> One place that all the AI calls in a firm go through.

**Field:** Software Development  
**Category:** AI Engineering  
**Also known as:** AI Gateway, LLM Proxy, Model Gateway

## Think of it like

Think of it like the front desk in a big building. Every guest signs in there first.

## Definition

An LLM gateway is a service that sits between your apps and the model makers, so every model call goes through one place. Apps call the gateway, and it sends each request on to the right model maker. It holds the API keys and sets limits on spend and on calls. It logs every call and its cost, and it can switch to another model maker when one fails. Teams use one when they have many apps or many models.

## Example

Emma's company has six apps that use AI. All of them call one LiteLLM gateway, so finance can see the full bill and security keeps the keys in one place.

## Why it matters

It gives one place to control cost, keys and logs, instead of each app doing it its own way.

## Common mistake

Adding a gateway for a single small app. It is one more thing to run, and it pays off only once there are many callers.

## Three principles

- **One way in.** For every call.
- **Keys and limits.** Held centrally.
- **Logs and costs.** In one place.

## Related terms

- [API Gateway](https://learn.benibix.com/term/development/api-gateway)
- [LLM API](https://learn.benibix.com/term/development/llm-api)
- [Model Fallback](https://learn.benibix.com/term/development/model-fallback)
- [LLM Observability](https://learn.benibix.com/term/development/llm-observability)
- [Rate Limiting](https://learn.benibix.com/term/development/rate-limiting)
- [OpenRouter](https://learn.benibix.com/term/ai/openrouter)
- [LiteLLM](https://learn.benibix.com/term/development/litellm)

## Told apart from

- [LLM Gateway vs API Gateway](https://learn.benibix.com/compare/llm-gateway-vs-api-gateway): People often think an LLM gateway is just an API gateway with a new name. Both sit in front of services and deal with keys, limits and logs. An API gateway guards your own services from outside callers. An LLM gateway sits in front of outside model makers for your own apps. It counts tokens, tracks the cost of each model and can switch model makers.

## Taught in

- [OpenRouter and Model APIs](https://learn.benibix.com/learn/ai-models-providers/openrouter-and-model-apis): AI Models and Providers, 13/13

![Several apps send their calls to one gateway. The gateway holds the keys, logs each call, and passes it on to one of several model providers.](https://learn.benibix.com/diagrams/dev-llm-gateway.svg)

---

Source: https://learn.benibix.com/term/development/llm-gateway
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-20
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
