⚙️Real-Time Inference
Also known as Online Inference, Online Prediction, Live Inference
A model runs the moment a request comes in. You wait for the answer.
Think of it like
Think of it like a crossing guard. They look and decide on the spot, while you wait at the curb.
Example
When Jack pays with his card, a fraud model scores the payment in about 50 milliseconds. The bank approves it before the shop's machine has finished beeping.
Why it matters
Some choices can't wait for tonight's batch. Real-time inference lets a model act on what is happening now.
Definition
Common mistake
Key takeaways
- 1Answer on request. One at a time.
- 2Fast. Milliseconds count.
- 3Always on. Costs more.