# vLLM

> A free tool that serves one AI model to lots of people at once.

**Field:** Artificial Intelligence  
**Category:** Running Locally  
**Also known as:** vLLM Server, vllm

## Think of it like

Think of it like a busy bus. It picks up many people on one trip, so no seat is wasted.

## Definition

vLLM is a free engine that serves language models from GPU servers to many users at once. It came out of a university lab in 2023. Its key idea, called PagedAttention, keeps each request's memory in small pages. So little space is wasted. It also adds new requests to the batch as old ones finish, which keeps the card busy. Its API has the same shape as common cloud model APIs, so apps can switch to it with ease.

## Example

Lucy's company hosts an open model for 2,000 staff. On the same GPU server, vLLM serves many more requests a second than a laptop tool could.

## Example code

```bash
vllm serve <model-name> --port 8000
```

Start a server for one model. Apps call it on port 8000.

## Why it matters

Serving many users is where the cost adds up. vLLM gets far more answers out of each graphics card.

## Common mistake

Using it for one person on a laptop. It is built for big servers and many users, and simpler tools fit a single user better.

## Three principles

- **Many users at once.** From one server.
- **Paged memory.** Little space wasted.
- **A familiar API.** Easy to switch.

## Related terms

- [Inference Engine](https://learn.benibix.com/term/ai/inference-engine)
- [Ollama](https://learn.benibix.com/term/ai/ollama)
- [KV Cache](https://learn.benibix.com/term/ai/kv-cache)
- [Graphics Processing Unit (GPU)](https://learn.benibix.com/term/infrastructure/graphics-processing-unit)
- [Throughput](https://learn.benibix.com/term/infrastructure/throughput)

## Told apart from

- [Ollama vs vLLM](https://learn.benibix.com/compare/ollama-vs-vllm): People often think vLLM is just a faster Ollama and the best pick for everyone. They are built for different jobs. Ollama makes it easy for one person to run a model on their own machine. vLLM is made to serve many users at once from GPU servers. It packs their requests together, so each card does more work.

![Many users send requests to one GPU server. vLLM packs their requests together and keeps the graphics card busy, so many answers come out at once.](https://learn.benibix.com/diagrams/lo-vllm.svg)

---

Source: https://learn.benibix.com/term/ai/vllm
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
