Skip to content
BENIBIX Learn

Model Serving

AI⚙️ MLOps & DeploymentModel Deployment

Also known as ML Model Serving, Model Deployment, Model Endpoint

A model put behind a web address. Apps send it data and get answers.

Think of it like

Think of it like a shop window with a bell. Anyone can ring, and someone comes to help.

Example

Emma's team serves a spam model at a web address. The email app sends each new message to it and gets back a spam score in about 20 milliseconds.

Why it matters

A model nobody can call is not doing any work. Serving is the step that puts it to use.

Definition

Model serving means putting a trained model behind a service that apps can call to get answers. A model server loads the model once. Then it waits for requests at a web address. An app sends input data, and the server runs the model and sends back the result. Serving also means keeping it fast, handling many calls at once, and swapping in new versions with no downtime. It is how a model leaves the lab and starts real work.

Example code

bash
curl -X POST http://localhost:8080/predict -d '{"text": "You won a prize!"}'
{"spam": 0.97}
Send one input to a served model and get its prediction back.
An app sends data to the model. The answer comes back in a few milliseconds.

Common mistake

Shipping a new model by replacing the old one all at once. Send a small share of traffic to it first, and watch.

Key takeaways

  1. 1
    Load once. Then wait for calls.
  2. 2
    Input in, answer out. At a web address.
  3. 3
    Swap safely. No downtime.