# AI Training and Inference

> How an AI model learns from examples in training, then answers in inference. Why training costs so much, and why each answer costs too.

**Course:** [AI Basics](https://learn.benibix.com/learn/ai-foundations)  
**Time:** 11 min

**After this lesson you can:**

- Tell training from inference
- Explain what training changes inside a model
- Say why AI needs so much computer power

## How a model learns, then answers

Every AI model has two stages. First comes training, when it learns from examples. Then comes inference, when it is used to answer. Training happens once and is slow. A famous photo model from 2012 took five to six days on two graphics chips. Inference happens each time someone asks, often in under a second.

Training starts with training data: thousands or millions of examples, each with the right answer. The model makes a guess for one example. The guess is checked against the answer. Then the model's numbers are nudged a tiny bit, so the next guess is closer. This repeats, again and again.

Those numbers are called parameters. They start out random, so the first guesses are nonsense. No one sets them by hand. Training does, one small nudge at a time. A small model may have a few thousand parameters. The largest have hundreds of billions.

Once training ends, the parameters are locked. Now the model is used for inference. A new photo goes in, and a label comes out. The model learns nothing from this. Asking it a question does not teach it your answer, which surprises a lot of people.

All of this needs compute: the raw power of the chips that do the sums. Training uses huge amounts of it at once. The compute used to train the top models has grown about four to five times each year since 2010. That is one reason AI labs spend so much on chips and power.

Inference costs far less per answer, but it never stops. A chat app may answer millions of questions a day, and each one runs the model once. So a company pays for training once and for inference all day long. For a busy app, that daily bill adds up fast.

The data shapes the model as much as the compute does. A model can only learn patterns that are in its examples. If the examples are wrong, too few or one-sided, the model will be too. More power does not fix poor data.

![Two rows. The top row, training, done once and slowly: many labelled examples go into a model whose numbers get nudged, each guess is checked against the answer, and the loop repeats. The bottom row, inference, done for every request: one new photo goes into the same locked model, which answers cat.](https://learn.benibix.com/diagrams/ai-training-inference.svg)

### Example: Jack trains an apple sorter

Jack's farm shop trains a model on 8,000 photos of apples, each marked ripe or not ripe. Training takes 3 hours on one rented graphics chip.

After that, each new photo takes a tenth of a second to check. His camera checks 600 apples a day, and the model itself never changes.

**Words in this lesson:** [Training (AI)](https://learn.benibix.com/term/ai/training), [Training Data](https://learn.benibix.com/term/ai/training-data), [Model Parameter](https://learn.benibix.com/term/ai/model-parameter), [Inference (AI)](https://learn.benibix.com/term/ai/inference), [AI Compute](https://learn.benibix.com/term/ai/ai-compute)

**Question:** Does an AI model learn from the questions you ask it?

**Answer:** No, its numbers stay locked while it answers. Inference uses a model whose numbers were fixed when training ended. Some apps keep chats to train a later model, but the model you talk to does not change.

---

Source: https://learn.benibix.com/learn/ai-foundations/ai-training-and-inference
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
