papers in adversarial machine learning — data poisoning

Generative AI models have become increasingly effective at making usable art. Where does this leave artists? They can use tools like Glaze and Nightshade to discourage others from training models to reproduce their art, but this might not always work, and carries legal risks. Here's how they work.

Maybe.

Data poisoning is when someone adds small changes to a training dataset to cause any model trained on those data to misbehave. An effective heuristic approach involves generating adversarial examples instead. The authors show degradations in model accuracy that are worse than random chance performance.

Using photographs of faces is becoming more and more common in automated identification systems, either for authentication or for surveillance. When these systems are based on machine learning models for face recognition, they are vulnerable to data poisoning attacks. By injecting as little as 50 watermarked images into the training set, you can force a model to misidentify you by putting on a physical accessory, like a pair of sunglasses.

Data poisoning attacks are very effective because they attack a model when it is most vulnerable, but poisoned images are expensive to compute. Here, we discuss two cheaper heuristics we can use -- feature alignment and watermarking -- how they work, and how effective they are at attacking computer vision systems.