Skip to content
BENIBIX Learn

LLM Gateway

Development🧬 AI EngineeringLLM Gateways

Also known as AI Gateway, LLM Proxy, Model Gateway

One place that all the AI calls in a firm go through.

Think of it like

Think of it like the front desk in a big building. Every guest signs in there first.

Example

Emma's company has six apps that use AI. All of them call one LiteLLM gateway, so finance can see the full bill and security keeps the keys in one place.

Why it matters

It gives one place to control cost, keys and logs, instead of each app doing it its own way.

Definition

An LLM gateway is a service that sits between your apps and the model makers, so every model call goes through one place. Apps call the gateway, and it sends each request on to the right model maker. It holds the API keys and sets limits on spend and on calls. It logs every call and its cost, and it can switch to another model maker when one fails. Teams use one when they have many apps or many models.
Every app calls the gateway. It holds the keys, logs the call and passes it to a provider.

Common mistake

Adding a gateway for a single small app. It is one more thing to run, and it pays off only once there are many callers.

Key takeaways

  1. 1
    One way in. For every call.
  2. 2
    Keys and limits. Held centrally.
  3. 3
    Logs and costs. In one place.

Compare

Learn it properly