Skip to content
BENIBIX Learn

Training-Serving Skew vs Data Drift

People think training-serving skew is just data drift, and it is not. Drift is the world changing after the model was trained, so the live data slowly moves away. Skew is your own system building the same input two ways, one for training and one for live use. Drift needs new data and a retrain. Skew needs a bug fix.

AI⚙️ MLOps & Deployment

Training-Serving Skew

Live data and training data are made in two ways. So a good model does badly.

Training-serving skew is when the data a model sees in real use differs from the data it was trained on, because of how the system was built. Often two pieces of code compute the same input in different ways. For example, training may use prices in dollars while the live app sends cents. The model still answers, but the answers are worse. It is a bug in the system, not a change in the world.

Example
Emma's model was trained on ages in years. The live app sends ages in months, so every customer looks 12 times older and the scores go wrong.
Why it matters
Nothing crashes, so it can hide for months. A model that tested well quietly gives poor answers.
Full entry
AI⚙️ MLOps & Deployment

Data Drift

The data that comes in starts to look less like what the model learned on.

Data drift is when a model's live inputs drift away from its training data. The model was tuned for one mix of inputs, such as the ages, places or phones of its users. Now that mix has moved. The model may still be right about the cases it knows, but it sees more cases it learned little about. You find drift by comparing live inputs with training inputs over time. It is often the first sign that a model needs new training.

Example
Jack's music app trained its model on users who were mostly in their 20s. A year later, after a TV ad, most new users are over 40, and the suggestions get worse.
Why it matters
It warns you early, from the inputs alone. The true answers come later and show the harm.
Full entry

In one line

Training-Serving Skew
Live data and training data are made in two ways. So a good model does badly.
Data Drift
The data that comes in starts to look less like what the model learned on.