Data Poisoning vs Adversarial Example
People think data poisoning and adversarial examples are one attack. They are not. Poisoning strikes while the model is still learning. Bad examples go into what it learns from, and the harm is built in. An adversarial example strikes once the model is done. The model is fine. One input is changed so the model reads it wrong.
Data Poisoning
An attack that hides bad samples in what an AI learns from.
Data poisoning is an attack that slips bad samples into the data a model learns from. Models learn from huge piles of text taken from the web, so an attacker can post bad pages and wait. The bad data can make the model worse, or plant a backdoor. With a backdoor, the model acts as usual until it sees a secret word. Then it does what the attacker wants. Studies show a few hundred bad files can be enough, even for a big model.
Adversarial Example
A tiny change made on purpose, so an AI gets it wrong.
An adversarial example is an input changed on purpose, often by a tiny bit, so a model gets it wrong. To a person, the photo or sound seems the same. But the change aims at the model's weak spots. So the answer flips, and the model is often very sure of it. In a famous case, faint noise was added to a photo of a panda, and the model said gibbon. Some work in the real world too, such as stickers on a stop sign that fool a camera.
In one line
- Data Poisoning
- An attack that hides bad samples in what an AI learns from.
- Adversarial Example
- A tiny change made on purpose, so an AI gets it wrong.