Data Drift vs Concept Drift
People often think data drift and concept drift are two names for one problem. In data drift, the inputs change: the model sees new kinds of cases it was not trained on. In concept drift, the right answer for the same input changes, so the model's rules go out of date. Checking inputs finds the first. Only real results show the second.
Data Drift
The data that comes in starts to look less like what the model learned on.
Data drift is when a model's live inputs drift away from its training data. The model was tuned for one mix of inputs, such as the ages, places or phones of its users. Now that mix has moved. The model may still be right about the cases it knows, but it sees more cases it learned little about. You find drift by comparing live inputs with training inputs over time. It is often the first sign that a model needs new training.
Concept Drift
The right answer for the same input changes. So a model's rules go out of date.
Concept drift is when the link between inputs and the right answer changes. Then the rules a model learned stop being true. The inputs can look just the same as before, but the right output for them is now different. Fraud is a common case, since people who cheat change their tricks once they are caught. How people shop after a big event is another. You can only see concept drift clearly by comparing the model's answers with the real results as they come in.
In one line
- Data Drift
- The data that comes in starts to look less like what the model learned on.
- Concept Drift
- The right answer for the same input changes. So a model's rules go out of date.