# Model Serving

> A model put behind a web address. Apps send it data and get answers.

**Field:** Artificial Intelligence  
**Category:** MLOps & Deployment  
**Also known as:** ML Model Serving, Model Deployment, Model Endpoint

## Think of it like

Think of it like a shop window with a bell. Anyone can ring, and someone comes to help.

## Definition

Model serving means putting a trained model behind a service that apps can call to get answers. A model server loads the model once. Then it waits for requests at a web address. An app sends input data, and the server runs the model and sends back the result. Serving also means keeping it fast, handling many calls at once, and swapping in new versions with no downtime. It is how a model leaves the lab and starts real work.

## Example

Emma's team serves a spam model at a web address. The email app sends each new message to it and gets back a spam score in about 20 milliseconds.

## Example code

```bash
curl -X POST http://localhost:8080/predict -d '{"text": "You won a prize!"}'
{"spam": 0.97}
```

Send one input to a served model and get its prediction back.

## Why it matters

A model nobody can call is not doing any work. Serving is the step that puts it to use.

## Common mistake

Shipping a new model by replacing the old one all at once. Send a small share of traffic to it first, and watch.

## Three principles

- **Load once.** Then wait for calls.
- **Input in, answer out.** At a web address.
- **Swap safely.** No downtime.

## Related terms

- [Model Registry](https://learn.benibix.com/term/ai/model-registry)
- [Real-Time Inference](https://learn.benibix.com/term/ai/real-time-inference)
- [Batch Inference](https://learn.benibix.com/term/ai/batch-inference)
- [API Endpoint](https://learn.benibix.com/term/development/endpoint)
- [Canary Release](https://learn.benibix.com/term/development/canary-release)
- [Provisioned Throughput](https://learn.benibix.com/term/ai/provisioned-throughput)
- [Open Neural Network Exchange (ONNX)](https://learn.benibix.com/term/ai/onnx)

![An app sends data to a model behind a web address. The model server runs the model and sends back a prediction in a few milliseconds.](https://learn.benibix.com/diagrams/mo-model-serving.svg)

---

Source: https://learn.benibix.com/term/ai/model-serving
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-17
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
