Abstract
Deep neural networks (DNN) have achieved unprecedented success in numerous machine learning tasks in various domains. However, the existence of adversarial examples raises our concerns in adopting deep learning to safety-critical applications. As a result, we have witnessed increasing interests in studying attack and defense mechanisms for DNN models on different data types, such as images, graphs and text. Thus, it is necessary to provide a systematic and comprehensive overview of the main threats of attacks and the success of corresponding countermeasures. In this survey, we review the state of the art algorithms for generating adversarial examples and the countermeasures against adversarial examples, for three most popular data types, including images, graphs and text.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
A. Krizhevsky, I. Sutskever, G. E. Hinton. Imagenet classification with deep convolutiona neura networks In Proceedings of the 25th International Conference on Neural Information Processing Systems, Curran Associates Inc., Lake Tahoe, USA, pp. 1097–1105, 2012.
K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE Las Vegas USA pp. 770–778, 2016 DOI 10.1109/CVPR.2016.90
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. R. Mohamed, N. Jaitly A. Senior, V. Vanhoucke P. Nguyen, T. N. Sainath, B. Kingsbury. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, vol. 29, no. 6, pp 82–97, 2012 DOI: 10.1109/MSP2012. 2205597.
S. Hochreiter, J. Schmidhuber. Long short-term memory. Neural Computation, vo. 9, no 8, pp 1735–1780, 1997 DOI: 10.1162/neco.1997.9.8.1735.
D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Dressche J. Schrittwieser I. Antonoglou V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N Kalchbrenner I. Sutskever T. Lillicrap M. Leach, K. Kavukcuoglu, T. Graepel, D. Hassabis. Mastering the game of go with deep neural networks and tree search. Nature, vol. 529, no. 7587, pp. 484–489, 2016. DOI: 10.1038/nature16961.
D. Cireçan, U. Meier, J. Masci, J. Schmidhuber. Multi-column deep neural network for traffic sign classification. Neural Networks, vol. 32, p. 333–338, 012. DO 10.1016/j.neunet.2012.02.023.
T. N Kipf, M. Weling. Sem-supervsed cassification with graph convolutional networks. ArXiv: 1609.02907, 2016.
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, R. Fergus Intriguing properties of neural networks. ArXiv: 1312.6199, 2013.
I. J. Goodfellow, J. Shlens, C. Szegedy. Explaining and harnessing adversarial examples. ArXiv: 1412.6572, 2014.
D. Zügner, A. Akbarnejad, S. Günnemann. Adversarial attacks on neural networks for graph data}. In Proceedings of the 24th ACM SIGKDD International Conference ton Knowledge Discovery & Data Mining, ACM, London, UK, p. 2847–2856, 018. OI:.1145/3219819. 3220078.
J. Ebrahimi, A. Y. Rao, D. Lowd, D. J. Dou. HotFlip: White-box adversaral exampes for text cassification ArXiv: 1712.06751, 2017.
N. Papernot, P. McDaniel, X. Wu, S. Jha, A. Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In Proceedings of IEEE Symposium on Security and Privacy, IEEE San Jose USA pp. 582–597, 2016. DOI: 10.1109/SP.2016.41.
A. Athalye, N. Carlini, D. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. ArXiv: 1802.00420, 2018.
A. Madry, A. Makelov, L. Schmidt, D. Tsipras, A. Vladu. Towards deep learning models resistant to adversarial attacks. ArXiv: 1706.06083, 2017.
A. Kurakn, I. Goodfelow, S. Bengo. Adversaral examples in the physical world. ArXiv: 1607.02533, 2016.
N. Carlini, D. Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, ACM, Dallas, USA, pp. 3–14, 2017. DOI: 10.1145/3128572.3140444.
W. L. Xu, D. Evans, Y. J. Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. ArXiv: 1704.01155, 2017.
A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, A. Madry. Adversarial examples are not bugs, they are features. ArXiv: 1905.02175, 2019.
B. Bggio, B. Neson P. Lakov. Posonng atacks against support vector machines. In Proceedings of the 29th International Coference on International Conference on Machine Learning, Omnipress, Edinburgh, UK, 2012.
K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. W. Xiao, A. Prakash, T. Kohno, D. Song. Robust physical-world attacks on deep learning models. ArXiv: 1707.08945, 2017.
F. Tramer, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, P. McDaniel. Ensemble adversarial training: Attacks and defenses. ArXiv: 1705.07204, 2017.
B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndić, P. Laskov, G. Giacinto, F. Roli. Evasion attacks against machine learning at test time. In Proceedings of European Conference on Machine Learning and Knowledge Discovery in Databases, Springer Prague Czech Repubic, pp. 387–402, 2013. DOI: 10.1007/978-3-642-40994-325.
M. Barreno, B. Nelson, A. D. Joseph, J. D. Tygar. The security of machne earning Machine Learning, vo. 81, no. 2, pp. 121–148, 2010. DOI: 10.1007/s10994-010-5188-5.
N. Dalvi, P. Domingos, Mausam, S. Sanghai, D. Verma. Adversarial cassification. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM eattle, USA, pp. 99–108, 2004. DOI: 10.1145/1014052.1014066.
D. Tspras, S. Santurkar, L. Engstrom, A. Turner, A. Madry. Robustness may be at odds with accuracy. ArXiv: 1805.12152, 2018.
D. Su, H. Zhang, H. G. Chen, J. F. Yi, P. Y. Chen, Y. P. Gao. Is robustness the cost of accuracy? - A comprehensive study on the robustness of 18 deep image classification models. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 644–661, 2018. DOI: 10.1007/978-3-030-01258-839.
D. Stutz, M. Hein, B. Schiele. Disentangling adversarial robustness and generalization. In Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Piscataway, USA, pp. 6976–6987, 2019.
H. Y. Zhang Y. D. Yu, J. T. Jiao E. P. Xing, L. El Ghaoui, M. I. Jordan. Theoretically principled trade-off between robustness and accuracy, ArXv: 190108573, 2019.
J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, F. F. Li. Imagenet: A large-scale hierarchical image database. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE Miam, USA pp. 248–255, 2009. DOI: 10.1109/CVPR.2009.5206848.
D. C. Liu, J. Nocedal. On the limted memory BFGS method for large scale optimization. Mathematical Programming, vol. 45, no. 1-3, p. 503–528, 1989. DO: 10.1007/BF01589116.
A. Kurakin, I. Goodfellow, S. Bengio. Adversarial machine learning at scale. ArXiv: 1611.01236, 2016.
S. M. Moosavi-Dezfooli, A. Fawzi, P. Frossard. DeepFool: A simple and accurate method to fool deep neural networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 2574–2582, 2016. DOI: 10.1109/CVPR.2016.282.
N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, A. Swami. The limitations of deep learning in adversarial settings. In Proceedings of IEEE European Symposium on Security and Privacy, IEEE Saarbrucken Germany, p. 372–387, 2016. DO: 0.1109/EuroSP. 2016.36.
N. Carlini, D. Wagner. Towards evaluating the robustness of neural networks. In Proceedings of IEEE Symposium on Security and Privacy, EEE, San Jose USA pp. 39–57, 2017. DOI: 10.1109/SP.2017.49.
N. Carlini, G. Katz, C. Barrett, D. L. Dill. Provably minimally-distorted adversarial examples. ArXiv: 1709.10207, 2017.
G. Kaz, C. Barett, D. L. Dill, K. Juian M. J. Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural networks. In Proceedings of the 29th International Conference on Computer Aided Verification, Springer, Heidelberg, Germany pp. 97–117, 2017. DOI: 10.1007/978-3-319-63387-9_5.
V. Tjeng, K. Xiao, R. Tedrake. Evaluating robustness of neural networks with mixed integer programming. ArXiv: 1711.07356, 2017.
K. Y. Xiao, V. Tjeng, N. M. Shafiullah, A. Madry. Training for faster adversarial robustness verification via inducing ReLU stability. ArXiv: 1809.03008, 2018.
J. W. Su, D. V. Vargas, K. Sakurai. One pixel attack for fooling deep neura networks IEEE Transactions on Evolutionary Computation, vol 23, no. 5, pp 828–841, 2019. DOI: 10.1109/TEVC.2019.2890858.
P. Y. Chen, Y. Sharma, H. Zhang, J. F. Yi, C. J. Hsieh. EAD: Elasticnet attacks to deep neural networks via adversarial exampes In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
Y. Sharma, P. Y. Chen. Attacking the madry defense model with L1-based dversarial xamples. ArXiv: 1710.10733, 2017.
S. M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, P. Frossard. Universal adversaral perturbatons In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Honolulu, USA, pp. 86–94, 2017. DOI: 10.1109/CVPR.2017.17.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. A. Ma, Z. H. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, F. F. Li. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, vol. 115, no. 3, p. 211–252, 15. DOI: 10.1007/s11263-015-0816-y.
C. W. Xiao, J. Y. Zhu, B. Li, W. He, M. Y. Liu, D. Song. Spatially ransformed dversarial xamples. ArXiv: 1801.02612, 2018.
Y. Song, R. Shu, N. Kushman, S. Ermon. Constructing unrestricted adversarial examples with generative models. In Proceedings of the 32nd Conference on Neural Information Processing Systems, Mntréal, Canada, pp. 8312–8323, 2018.
A. Odena, C. Olah, J. Shlens. Conditional image synthesis with auxiliary classifier GANs. In Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, pp. 2642–2651, 2017.
A. Athalye, L. Engstrom, A. Ilyas, K. Kwok. Synthesizing robust adversarial examples. ArXiv: 1707.07397, 2017.
N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, A. Swami. Practical black-box attacks against machine learning. In Proceedings of ACM on Asia Conference on Computer and Communications Security, ACM, Abu Dhab, Unted Arab Emrates pp. 506–519 2017 DOI: 10.1145/3052973.3053009.
Y. P. Dong, F. Z. Liao, T. Y. Pang, H. Su, J. Zhu, X. L. Hu, J. G. Li. Boosting adversaral attacks with momentum. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake Cty USA pp. 9185–9193 2018 DOI: 101109/ CVPR.2018.00957.
P. Y. Chen, H. Zhang, Y. Sharma, J. F. Yi, C. J. Hsieh. ZOO: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, ACM Dallas, USA, pp. 15–26, 2017. DOI: 10.1145/3128572.3140448.
A. Ilyas, L. Engstrom, A. Athalye, J. Lin. Black-box adversarial attacks with limited queries and information. ArXiv: 1804.08598, 2018.
D. Werstra T. Schau, T. Glasmachers, Y. Sun, J. Peters, J. Schmdhuber. Natural evouton strateges Natural evolution strategies. Journal of Machine Learning Research, vol. 15, no. 1, pp. 949–980, 2014.
M. Alzantot, Y. Sharma, S. Chakraborty, M. Srivastava. Genattack: Practical black-box attacks with gradient-free optimization. ArXiv: 1805.11090, 2018.
C. W. Xiao, B. Li, J. Y. Zhu, W. He, M. Y. Liu, D. Song. Generating adversarial examples with adversarial networks. ArXiv: 1801.02610, 2018.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems, MIT Press, Montreal, Canada, pp. 2672–2680, 2014.
D. Deb, J. B. Zhang, A. K. Jain. Advfaces: Adversarial face synthesis. ArXiv: 1908.05008, 2019.
G. Cauwenberghs, T. Poggio. Incremental and decre- mental support vector machine learning. In Proceedings of the 13th International Conference on Neural Information Processing Systems, MIT Press, Denver, USA, pp. 388–394, 2000.
P. W. Koh, P. Liang. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, pp. 1885–1894, 2017.
A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer, T. Dumitras, T. Goldstein. Poison frogs! Targeted clean- label poisoning attacks on neural networks. In Proceedings of the 32nd Conference on Neural Information Processing Systems, Montreal, Canada, pp. 6103–6113, 2018.
G. Hinton, O. Vinyals, J. Dean. Distilling the knowledge in a neural network. ArXiv: 1503.02531, 2015.
J. Buckman, A. Roy, C. Raffel, I. Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada, 2018.
C. Guo, M. Rana, M. Cisse, L. van der Maaten. Countering adversarial images using input transformations. ArXiv: 1711.00117, 2017.
V. K. Ha, J. C. Ren, X. Y. Xu, S. Zhao, G. Xie, V. M. Vargas. Deep learning based single image super-resolution: A survey. In Proceedings of the 9th International Conference on Brain Inspired Cognitive Systems, Springer, Xi'an, China, pp. 106–119, 2018. DOI: 10.1007/978-3-030-00563-4_11.
G. S. Dhillon, K. Azizzadenesheli, Z. C. Lipton, J. Bernstein, J. Kossaifi, A. Khanna, A. Anandkumar. Stochastic activation pruning for robust adversarial defense. ArXiv: 1803.01442, 2018.
C. H. Xie, J. Y. Wang, Z. S. Zhang, Z. Ren, A. Yuille. Mitigating adversarial effects through randomization. ArXiv: 1711.01991, 2017.
Y. Song, T. Kim, S. Nowozin, S. Ermon, N. Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. ArXiv: 1710.10766, 2017.
P. Samangouei, M. Kabkab, R. Chellappa. Defense-GAN: Protecting classifiers against adversarial attacks using generative models. ArXiv: 1805.06605, 2018.
A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves, K. Kavukcuoglu. Conditional image generation with PixelCNN decoders. In Proceedings of the 30th Conference on Neural Information Processing Systems, Curran Associates Inc., Barcelona, Spain, pp. 4790–4798, 2016.
M. Cisse, P. Bojanowski, E. Grave, Y. Dauphin, N. Usunier. Parseval networks: Improving robustness to adversarial examples. In Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, pp. 854–863, 2017.
T. Miyato, S. I. Maeda, M. Koyama, K. Nakae, S. Ishii. Distributional smoothing with virtual adversarial training. ArXiv: 1507.00677, 2015.
S. X. Gu, L. Rigazio. Towards deep neural network architectures robust to adversarial examples. ArXiv: 1412.5068, 2014.
S. Rifai, P. Vincent, X. Muller, X. Glorot, Y. Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedings of the 28th International Conference on International Conference on Machine Learning, Omnipress, Bellevue, USA, pp. 833–840, 2011.
S. Ioffe, C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ArXiv: 1502.03167, 2015.
A. Shafahi, M. Najibi, A. Ghiasi, Z. Xu, J. Dickerson, C. Studer, L. S. Davis, G. Taylor, T. Goldstein. Adversarial training for free. ArXiv: 1904.12843, 2019.
D. H. Zhang, T. Y. Zhang, Y. P. Lu, Z. X. Zhu, B. Dong. You only propagate once: Accelerating adversarial training via maximal principle. ArXiv: 1905.00877, 2019.
L. S. Pontryagin. Mathematical Theory of Optimal Processes, London, UK: Routledge, 2018.
A. Raghunathan, J. Steinhardt, P. Liang. Certified defenses against adversarial examples. ArXiv: 1801.09344, 2018.
E. Wong, J. Z. Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. ArXiv: 1711.00851, 2017.
M. Hein, M. Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, USA, pp. 2266–2276, 2017.
L. Vandenberghe, S. Boyd. Semidefinite programming. Semidefinite programming. SIAM Review, vol. 38, no. 1, pp. 49–95, 1996. DOI: 10.1137/1038003.
A. Raghunathan, J. Steinhardt, P. S. Liang. Semidefinite relaxations for certifying robustness to adversarial examples. In Proceedings of the 32nd Conference on Neural Information Processing Systems, Montreal, Canada, pp. 10877–10887, 2018.
E. Wong, F. Schmidt, J. H. Metzen, J. Z. Kolter. Scaling provable adversarial defenses. In Proceedings of the 32nd Conference on Neural Information Processing Systems, Montreal, Canada, pp. 8400–8409, 2018.
A. Sinha, H. Namkoong, J. Duchi. Certifying some distributional robustness with principled adversarial training. ArXiv: 1710.10571, 2017.
K. Grosse, P. Manoharan, N. Papernot, M. Backes, P. McDaniel. On the (statistical) detection of adversarial examples. ArXiv: 1702.06280, 2017.
Z. T. Gong, W. L. Wang, W. S. Ku. Adversarial and clean data are not twins. ArXiv: 1704.04960, 2017.
J. H. Metzen, T. Genewein, V. Fischer, B. Bischoff. On detecting adversarial perturbations. ArXiv: 1702.04267, 2017.
D. Hendrycks, K. Gimpel. Early methods for detecting adversarial images. ArXiv: 1608.00530, 2016.
A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Scholkopf, A. Smola. A kernel two-sample test. A kernel two-sample test. Journal of Machine Learning Research, vol. 13, pp. 723–773, 2012.
R. Feinman, R. R. Curtin, S. Shintre, A. B. Gardner. Detecting adversarial samples from artifacts. ArXiv: 1703.00410, 2017.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.
Y. Sharma, P. Y. Chen. Bypassing feature squeezing by increasing adversary strength. ArXiv: 1803.09868, 2018.
A. Fawzi, S. M. Moosavi-Dezfooli, P. Frossard. Robustness of classifiers: From adversarial to random noise. In Proceedings of the 30th Conference on Neural Information Processing Systems, Barcelona, Spain, pp. 1632–1640, 2016.
S. M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, P. Frossard, S. Soatto. Analysis of universal adversarial perturbations. ArXiv: 1705.09554, 2017.
A. Fawzi, O. Fawzi, P. Frossard. Analysis of classifiers' robustness to adversarial perturbations. Machine Learning, vol. 107, no. 3, pp. 481–508, 2018. DOI: 10.1007/ s10994-017-5663-3.
A. Shafahi, W. R. Huang, C. Studer, S. Feizi, T. Goldstein. Are adversarial examples inevitable. ArXiv: 1809.02104, 2018.
L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar, A. Madry. Adversarially robust generalization requires more data. In Proceedings of the 32nd Conference on Neural Information Processing Systems, Montreal, Canada, pp. 5014–5026, 2018.
H. J. Dai, H. Li, T. Tian, X. Huang, L. Wang, J. Zhu, L. Song. Adversarial attack on graph structured data. ArXiv: 1806.02371, 2018.
Y. Ma, S. H. Wang, T. Derr, L. F. Wu, J. L. Tang. Attacking graph convolutional networks via rewiring. ArXiv: 1906.03750, 2019.
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller. Playing Atari with deep reinforcement learning. ArXiv: 1312.5602, 2013.
D. Zuugner, S. Gunnemann. Adversarial attacks on graph neural networks via meta learning. ArXiv: 1902.08412, 2019.
C. Finn, P. Abbeel, S. Levine. Model-agnostic metalearning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, JMLR.org, Sydney, Australia, pp. 1126–1135, 2017.
A. Bojchevski, S. Gunnemann. Adversarial attacks on node embeddings via graph poisoning. ArXiv: 1809.01093, 2018.
B. Perozzi, R. Al-Rfou, S. Skiena. DeepWalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, New York, USA, pp. 701–710, 2014. DOI: 10.1145/2623330.2623732.
F. L. Feng, X. N. He, J. Tang, T. S. Chua. Graph adversarial training: Dynamically regularizing based on graph structure. ArXiv: 1902.08226, 2019.
K. D. Xu, H. G. Chen, S. J. Liu, P. Y. Chen, T. W. Weng, M. Y. Hong, X. Lin. Topology attack and defense for graph neural networks: An optimization perspective. ArXiv: 1906.04214, 2019.
N. Carlini, D. Wagner. Audio adversarial examples: Targeted attacks on speech-to-text. In Proceedings of IEEE Security and Privacy Workshops, IEEE, San Francisco, USA, pp. 1–7, 2018. DOI: 10.1109/SPW.2018.00009.
A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates, A. Y. Ng. Deep speech: Scaling up end-to-end speech recognition. ArXiv: 1412.5567, 2014.
T. Miyato, A. M. Dai, I. Goodfellow. Adversarial training methods for semi-supervised text classification. ArXiv: 1605.07725, 2016.
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems, Curran Associates Inc., Lake Tahoe, USA, pp. 3111–3119, 2013.
B. Liang, H. C. Li, M. Q. Su, P. Bian, X. R. Li, W. C. Shi. Deep text classification can be fooled. ArXiv: 1704.08006, 2017.
J. Gao, J. Lanchantin, M. L. Soffa, Y. J. Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In Proceedings of IEEE Security and Privacy Workshops, IEEE, San Francisco, USA, pp. 50–56, 2018. DOI: 10.1109/SPW.2018.00016.
J. F. Li, S. L. Ji, T. Y. Du, B. Li, T. Wang. Textbugger: Generating adversarial text against real-world applications. ArXiv: 1812.05271, 2018.
S. Samanta, S. Mehta. Towards crafting text adversarial samples. ArXiv: 1707.02812, 2017.
M. Iyyer, J. Wieting, K. Gimpel, L. Zettlemoyer. Adversarial example generation with syntactically controlled paraphrase networks. ArXiv: 1804.06059, 2018.
Q. Lei, L. F. Wu, P. Y. Chen, A. G. Dimakis, I. S. Dhillon, M. Witbrock. Discrete attacks and submodular optimization with applications to text classification. ArXiv: 1812.00151, 2018.
R. Jia, P. Liang. Adversarial examples for evaluating reading comprehension systems. ArXiv: 1707.07328, 2017.
Y. Belinkov, Y. Bisk. Synthetic and natural noise both break neural machine translation. ArXiv: 1711.02173, 2017.
M. H. Cheng, J. F. Yi, H. Zhang, P. Y. Chen, C. J. Hsieh. Seq2Sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples. ArXiv: 1803.01128, 2018.
T. Niu, M. Bansal. Adversarial over-sensitivity and overstability strategies for dialogue models. ArXiv: 1809.02079, 2018.
T. X. He, J. Glass. Detecting egregious responses in neural sequence-to-sequence models. ArXiv: 1809.04113, 2018.
H. C. Liu, T. Derr, Z. T. Liu, J. L. Tang. Say what I want: Towards the dark side of neural dialogue models. ArXiv: 1909.06044, 2019.
M. Sharif, S. Bhagavatula, L. Bauer, M. K. Reiter. Accessorize to a crime: Real and stealthy attacks on state-of- the-art face recognition. In Proceedings of the ACM SIG- SAC Conference on Computer and Communications Security, ACM, Vienna, Austria, pp. 1528–1540, 2016. DOI: 10.1145/2976749.2978392.
O. M. Parkhi, A. Vedaldi, A. Zisserman. Deep face recognition. Machine Learning, 2015.
C. H. Xie, J. Y. Wang, Z. S. Zhang, Y. Y. Zhou, L. X. Xie, A. Yuille. Adversarial examples for semantic segmentation and object detection. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Venice, Italy, pp. 1378–1387, 2017. DOI: 10.1109/ICCV.2017. 153.
J. H. Metzen, M. C. Kumar, T. Brox, V. Fischer. Universal adversarial perturbations against semantic image segmentation. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Venice, Italy, pp. 2774–2783, 2017. DOI: 10.1109/ICCV.2017.300.
S. S. Li, A. Neupane, S. Paul, C. Y. Song, S. V. Krishnamurthy, A. K. R. Chowdhury, A. Swami. Adversarial perturbations against real-time video classification systems. ArXiv: 1807.00458, 2018.
J. Kos, I. Fischer, D. Song. Adversarial examples for generative models. In Proceedings of IEEE Security and Privacy Workshops, IEEE, San Francisco, USA, pp. 36–42, DOI: 10.1109/SPW.2018.00014.
D. P. Kingma, M. Welling. Auto-encoding variational Bayes. ArXiv: 1312.6114, 2013.
A. B. L. Larsen, S. K. Sønderby, H. Larochelle, O. Winther. Autoencoding beyond pixels using a learned similarity metric. ArXiv: 1512.09300, 2015.
K. Grosse, N. Papernot, P. Manoharan, M. Backes, P. McDaniel. Adversarial perturbations against deep neural networks for malware classification. ArXiv: 1606.04435, 2016.
D. Arp, M. Spreitzenbarth, H. Gascon, K. Rieck. DREBIN: Effective and explainable detection of android malware in your pocket. In Proceedings of Symposium Network Distributed System Security, Internet Society, San Diego, USA, 2014.
W. W. Hu, Y. Tan. Generating adversarial malware examples for black-box attacks based on GAN. ArXiv: 1702.05983, 2017.
H. S. Anderson, J. Woodbridge, B. Filar. DeepDGA: Ad- versarially-tuned domain generation and detection. In Proceedings of ACM Workshop on Artificial Intelligence and Security, ACM, Vienna, Austria, pp. 13–21, 2016. DOI: 10.1145/2996758.2996767.
T. Chugh, A. K. Jain. Fingerprint presentation attack detection: Generalization and efficiency. ArXiv: 1812.11574, 2018.
T. Chugh, K. Cao, A. K. Jain. Fingerprint spoof buster: Use of minutiae-centered patches. IEEE Transactions on Information Forensics and Security, vol. 13, no. 9, pp. 2190–2202, 2018. DOI: 10.1109/TIFS.2018.2812193.
S. Huang, N. Papernot, I. Goodfellow, Y. Duan, P. Abbeel. Adversarial attacks on neural network policies. ArXiv: 1702.02284, 2017.
J. Schulman, S. Levine, P. Moritz, M. I. Jordan, P. Abbeel. Trust region policy optimization. In Proceedings of the 31st International Conference on Machine Learning, JMLR, Lille, France, pp. 1889–1897, 2015.
V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Harley, T. P. Lillicrap, D. Silver, K. Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In Proceedings of the 33rd International conference on Machine Learning, PMLR, New York, USA, pp. 1928–1937, 2016.
Acknowledgements
This work was supported by National Science Foundation (NSF), USA (Nos. IIS-1845081 and CNS-1815636).
Author information
Authors and Affiliations
Corresponding author
Additional information
Recommended by Associate Editor Hong Qiao
Han Xu is a second year Ph. D. student of computer science in DSE Lab, Michigan State University, USA. He is under supervision by Dr. Ji-Liang Tang.
His research interests include deep learning safety and robustness, especially studying the problems related to adversarial examples.
Yao Ma received the B. Sc. degree in applied mathematics at Zhejiang University, China in 2015, the M. Sc. degree in statistics, probabilities and operation research at Eindhoven University of Technology, the Netherlands in 2016. He is now a Ph. D. degree candidate of Department of Computer Science and Engineering, Michigan State University, USA. His Ph. D. advisor is Dr. Jiliang Tang.
His research interests include graph neural networks and their related safety issues.
Hao-Chen Liu is currently a Ph. D. student at the Department of Computer Science and Engineering at Michigan State University, under the supervision of Dr. Jiliang Tang. He is a member of Data Science and Engineering (DSE) Lab.
His research interests include natural language processing problems, especially in the robustness, fairness of dialogue systems.
Debayan Deb is a Ph. D. degree candidate in the Biometrics Lab, Michigan State University, USA under the supervision of Dr. Anil K. Jain. Before joining the Biometrics Lab of MSU, he graduated from Michigan State University with a Bachelor Degree of Computer Science and Engineering.
His research interests include face recognition and computer vision tasks.
Hui Liu is a research associate at Michigan State University. Before joining MSU, she received her Ph. D. degree of Electrical Engineering in Southern Methodist University, USA under the supervision by Dr. Dinesh Rajen.
Her research interests include signal processing, wireless communication, and deep learning related topics.
Ji-Liang Tang is an assistant professor in the computer science and engineering department at Michigan State University since Fall 2016. Before that, he was a research scientist in Yahoo Research and received his Ph. D. degree from Arizona State University in 2015. He was the recipients of 2019 NSF Career Award, the 2015 KDD Best Dissertation runner up and 6 Best Paper Awards (or runner-ups) including WSDM 2018 and KDD 2016. He serves as conference organizers (e.g., KDD, WSDM and SDM) and journal editors (e.g., TKDD). He has published his research in highly ranked journals and top conference proceedings, which received thousands of citations and extensive media coverage.
His research interests include social computing, data mining and machine learning and their applications in education.
Anil K. Jain (Ph. D., 1973, Ohio State University; B. Tech., IIT Kanpur) is a University Distinguished Professor at Michigan State University where he conducts research in pattern recognition, machine learning, computer vision, and biometrics recognition. He was a member of the United States Defense Science Board and Forensics Science Standards Board. His prizes include Guggenheim, Humboldt, Fulbright, and King-Sun Fu Prize. For advancing pattern recognition, Jain was awarded Doctor Honoris Causa by Universidad Autónoma de Madrid. He was Editor-in-Chief of the IEEE Transactions on Pattern Analysis and Machine Intelligence and is a Fellow of ACM, IEEE, AAAS, and SPIE. Jain has been assigned 8 U.S. and Korean patents and is active in technology transfer for which he was elected to the National Academy of Inventors. Jain is a member of the U.S. National Academy of Engineering (NAE), foreign member of the Indian National Academy of Engineering (INAE), a member of The World Academy of Science (TWAS) and a foreign member of the Chinese Academy of Sciences (CAS).
His research interests include pattern recognition, machine learning, computer vision, and biometrics recognition.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
To view a copy of this licence, visit http://creativecomm-ons.org/licenses/by/4.0/.
About this article
Cite this article
Xu, H., Ma, Y., Liu, HC. et al. Adversarial Attacks and Defenses in Images, Graphs and Text: A Review. Int. J. Autom. Comput. 17, 151–178 (2020). https://doi.org/10.1007/s11633-019-1211-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-019-1211-x