Skip to content
BENIBIX Learn

LiteLLM

Development🧬 AI EngineeringLLM Gateways

Also known as LiteLLM Proxy, LiteLLM SDK, LiteLLM Gateway

A free tool that lets your code call over 100 models in one way.

Think of it like

It is like a travel plug. One plug fits the wall in every country.

Example

Jack's team runs the LiteLLM proxy. Each app gets its own key, and moving to another provider is a change to one model name.

Why it matters

Apps stop being tied to one provider. Trying a new model, or leaving an old one, is a change to settings instead of a rewrite.

Definition

LiteLLM is a free, open-source tool for calling over 100 language models in one way. You write each call in the OpenAI format. LiteLLM turns it into what each provider expects. You name the model with the provider first, like anthropic/ and then the model. It comes as a Python library, or as a proxy server a whole team can share. The proxy adds its own keys, cost tracking and an admin page.

Example code

python
from litellm import completion

response = completion(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Hello, how are you?"}],
)
print(response.choices[0].message.content)
The provider comes first in the model name. The reply has the same shape for every provider.
LiteLLM can be this middle box: one door your apps call for every model.

Common mistake

Thinking every model can do the same things. A shared format cannot give a model a skill it does not have.

Key takeaways

  1. 1
    One format. OpenAI style for every model.
  2. 2
    Library or proxy. In your code, or shared by a team.
  3. 3
    Provider first. anthropic/, openai/ and more.

Compare