papers in adversarial machine learning — adversarial trigger

Previous attacks on automated identity recognition systems used large and obvious physical accessories, like giant sunglasses. It's possible to use something more subtle -- like a specific facial expression -- to trick one of these systems into believing you are another person. However, you will need to have control of a large fraction of the photographs of interest to get a good attack success rate, which could be achievable inside "walled garden" image hosting websites like Facebook.

Using photographs of faces is becoming more and more common in automated identification systems, either for authentication or for surveillance. When these systems are based on machine learning models for face recognition, they are vulnerable to data poisoning attacks. By injecting as little as 50 watermarked images into the training set, you can force a model to misidentify you by putting on a physical accessory, like a pair of sunglasses.