Skip to content
BENIBIX Learn

Data Drift

AI⚙️ MLOps & DeploymentProduction Monitoring

Also known as Input Drift, Covariate Shift, Feature Drift

The data that comes in starts to look less like what the model learned on.

Think of it like

Think of it like a shoe shop whose customers slowly get bigger feet. The old sizes stop fitting.

Example

Jack's music app trained its model on users who were mostly in their 20s. A year later, after a TV ad, most new users are over 40, and the suggestions get worse.

Why it matters

It warns you early, from the inputs alone. The true answers come later and show the harm.

Definition

Data drift is when a model's live inputs drift away from its training data. The model was tuned for one mix of inputs, such as the ages, places or phones of its users. Now that mix has moved. The model may still be right about the cases it knows, but it sees more cases it learned little about. You find drift by comparing live inputs with training inputs over time. It is often the first sign that a model needs new training.
Training data peaks near age 30. A year later the live data peaks near 45.

Common mistake

Training again each time an input moves a bit. Some drift does no harm, so check if the scores really fell.

Key takeaways

  1. 1
    The inputs move. From training data.
  2. 2
    Compare over time. Live vs training.
  3. 3
    Early warning. Before accuracy shows it.

Compare