Skip to content
BENIBIX Learn

Data Leakage

Also known as Leakage, Target Leakage, Test Leakage

When the answers sneak into training. Test scores look great and real use fails.

Think of it like

Think of it like seeing the answers before a test. Your score is high, but you learned nothing.

Example

Jack's model predicts which patients will get sick. One feature is the date the patient got medicine. That date gives the answer away, so the model looks perfect and fails in real use.

Why it matters

It is one of the most common reasons a model that looked great fails after launch. Nothing warns you, because the scores look fine.

Definition

Data leakage is when facts the model should not have slip into its training. It might be test examples hiding in the training set. It might be a feature that gives away the answer. The model then scores very well in testing. Later, with real data, it does much worse.
The dashed arrow is data leakage. Test rows that reach training make the final score a lie.

Common mistake

Splitting the data after cleaning it all together. Steps like filling gaps should learn from the training set only.

Key takeaways

  1. 1
    Answers sneak in. Facts the model should not have.
  2. 2
    Scores too good. Testing looks better than reality.
  3. 3
    Fails for real. New data shows the truth.

Compare

Learn it properly