Skip to content
BENIBIX Learn

Data Poisoning vs Adversarial Example

People think data poisoning and adversarial examples are one attack. They are not. Poisoning strikes while the model is still learning. Bad examples go into what it learns from, and the harm is built in. An adversarial example strikes once the model is done. The model is fine. One input is changed so the model reads it wrong.

AI🛡️ Safety & Security

Data Poisoning

An attack that hides bad samples in what an AI learns from.

Data poisoning is an attack that slips bad samples into the data a model learns from. Models learn from huge piles of text taken from the web, so an attacker can post bad pages and wait. The bad data can make the model worse, or plant a backdoor. With a backdoor, the model acts as usual until it sees a secret word. Then it does what the attacker wants. Studies show a few hundred bad files can be enough, even for a big model.

Example
An attacker posts pages where a made-up trigger word is always followed by nonsense. A model trained on the web picks them up. Later, it prints nonsense whenever it sees that word.
Why it matters
Nobody can read every page a model trains on. A small, hidden change in the data can ship inside the model.
Full entry
AI🛡️ Safety & Security

Adversarial Example

A tiny change made on purpose, so an AI gets it wrong.

An adversarial example is an input changed on purpose, often by a tiny bit, so a model gets it wrong. To a person, the photo or sound seems the same. But the change aims at the model's weak spots. So the answer flips, and the model is often very sure of it. In a famous case, faint noise was added to a photo of a panda, and the model said gibbon. Some work in the real world too, such as stickers on a stop sign that fool a camera.

Example
Researchers put a few small stickers on a stop sign. People still read it as a stop sign, but a vision model reads it as a speed limit sign.
Why it matters
Vision and speech models are used in cars, security and medicine. A cheap trick that fools them is a real danger.
Full entry

In one line

Data Poisoning
An attack that hides bad samples in what an AI learns from.
Adversarial Example
A tiny change made on purpose, so an AI gets it wrong.