# Real-Time Inference

> A model runs the moment a request comes in. You wait for the answer.

**Field:** Artificial Intelligence  
**Category:** MLOps & Deployment  
**Also known as:** Online Inference, Online Prediction, Live Inference

## Think of it like

Think of it like a crossing guard. They look and decide on the spot, while you wait at the curb.

## Definition

Real-time inference runs a model the moment a request comes in. It sends back the answer while the user waits. Each request is scored on its own, most often in tens or hundreds of milliseconds. You need it when the answer depends on what is going on right now, such as a card payment or a search. It costs more than batch work, since the model has to be loaded and ready at all times. Speed limits often decide which models can be used at all.

## Example

When Jack pays with his card, a fraud model scores the payment in about 50 milliseconds. The bank approves it before the shop's machine has finished beeping.

## Why it matters

Some choices can't wait for tonight's batch. Real-time inference lets a model act on what is happening now.

## Common mistake

Making everything real time by default. If the answer can wait, a batch job is cheaper and simpler.

## Three principles

- **Answer on request.** One at a time.
- **Fast.** Milliseconds count.
- **Always on.** Costs more.

## Related terms

- [Batch Inference](https://learn.benibix.com/term/ai/batch-inference)
- [Model Serving](https://learn.benibix.com/term/ai/model-serving)
- [Feature Store](https://learn.benibix.com/term/ai/feature-store)
- [Latency](https://learn.benibix.com/term/infrastructure/latency)

## Told apart from

- [Batch vs Real-Time Inference](https://learn.benibix.com/compare/batch-inference-vs-real-time-inference): People often think real-time inference is just the better kind and batch is the old way. They fit different needs. Batch inference scores lots of data at set times and saves it. That is cheap, but only as fresh as the last run. Real-time inference answers each request as it comes. That is fresh, but costs more to keep running.

![A card payment goes to a fraud model and gets a score back before the payment is approved, all within a fraction of a second.](https://learn.benibix.com/diagrams/mo-real-time-inference.svg)

---

Source: https://learn.benibix.com/term/ai/real-time-inference
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-17
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
