Skip to content
BENIBIX Learn

Data Poisoning

Also known as Training Data Poisoning, Backdoor Attack, Poisoning Attack

An attack that hides bad samples in what an AI learns from.

Think of it like

It is like slipping a few wrong pages into a schoolbook. Kids learn the wrong thing and do not notice.

Example

An attacker posts pages where a made-up trigger word is always followed by nonsense. A model trained on the web picks them up. Later, it prints nonsense whenever it sees that word.

Why it matters

Nobody can read every page a model trains on. A small, hidden change in the data can ship inside the model.

Definition

Data poisoning is an attack that slips bad samples into the data a model learns from. Models learn from huge piles of text taken from the web, so an attacker can post bad pages and wait. The bad data can make the model worse, or plant a backdoor. With a backdoor, the model acts as usual until it sees a secret word. Then it does what the attacker wants. Studies show a few hundred bad files can be enough, even for a big model.
A few bad samples in a big pile, and a model that waits for a trigger.

Common mistake

Thinking a big pile of data waters down the poison. A backdoor can work with very few bad samples, no matter how much clean data there is.

Key takeaways

  1. 1
    Bad data in. Before training.
  2. 2
    Backdoor. A hidden trigger.
  3. 3
    Few examples. Can be enough.

Compare