Skip to content
BENIBIX Learn

Training vs Inference

People often think a chatbot learns from them while they talk. It does not. Training is when a model learns, and it is over before the model is shared. Inference is each time the model answers you. The model stays the same then, so your chat does not teach it.

AI🧭 Foundations

Training (AI)

The learning stage of AI. A model practises on examples until its guesses get good.

Training is how an AI model learns. The model sees many examples and makes a guess for each one. Its guess is checked against the right answer. Then its numbers are nudged so the next guess is a little closer. This repeats millions of times until the guesses are good. Big models can take weeks to train on many powerful chips.

Example
Jack trains a model on 10,000 photos of cats and dogs, each labelled. After training, it can sort new photos it has never seen.
Why it matters
Training is the slow and costly part. Once it is done, the model can be used again and again for little cost.
Full entry
AI🧭 Foundations

Inference (AI)

Using a trained model to get an answer. It does not learn while it works.

Inference is using a trained AI model to get an answer. You give the model a new input, and it gives back an output. The model does not learn anything new while it does this. Each answer costs a little computer time, so busy apps pay for inference all day. Speed matters here, because people are waiting for the reply.

Example
Each time Emma asks a chatbot a question, the company runs the model once. A million questions a day means a million runs.
Why it matters
Training happens once. Inference happens each time someone uses the model, so that is where most of the running cost is.
Full entry

In one line

Training (AI)
The learning stage of AI. A model practises on examples until its guesses get good.
Inference (AI)
Using a trained model to get an answer. It does not learn while it works.