research-article

Open access

A Smart Adversarial Attack on Deep Hashing Based Image Retrieval

Authors:

Xiaochun YangAuthors Info & Claims

ICMR '21: Proceedings of the 2021 International Conference on Multimedia Retrieval

Pages 227 - 235

https://doi.org/10.1145/3460426.3463640

Published: 01 September 2021 Publication History

Abstract

Deep hashing based retrieval models have been widely used in large-scale image retrieval systems. Recently, there has been a surging interest in studying the adversarial attack problem in deep hashing based retrieval models. However, the effectiveness of existing adversarial attacks is limited by their poor perturbation management, unawareness of ranking weight, and only laser-focusing on the attack image. These shortages lead to high perturbation costs yet low AP reductions. To overcome these shortages, we propose a novel adversarial attack framework to improve the effectiveness of adversarial attacks. Our attack designs a dimension-wise surrogate Hamming distance function to help with wiser perturbation management. Further, in generating adversarial examples, instead of focusing on a single image, we propose to collectively incorporate relevant images combined with an AP-oriented (average precision) weight function. In addition, our attack can deal with both untargeted and targeted adversarial attacks in a flexible manner. Extensive experiments demonstrate that, with the same attack performance, our model significantly outperforms state-of-the-art models in perturbation cost on both untargeted and targeted attack tasks.

References

[1]

Laurent Amsaleg, James Bailey, Amélie Barbe, Sarah M. Erfani, Teddy Furon, Michael E. Houle, Milo? Radovanovi?, and Xuan Vinh Nguyen. 2021. High Intrinsic Dimensionality Facilitates Adversarial Attack: Theoretical Evidence. IEEE Transactions on Information Forensics and Security, Vol. 16 (2021), 854--865. https://doi.org/10.1109/TIFS.2020.3023274

[2]

L. Amsaleg, J. Bailey, Dominique Barbe, S. Erfani, M. Houle, Vinh Nguyen, and M. Radovanović. 2017. The vulnerability of learning to adversarial perturbation increases with intrinsic dimensionality. 2017 IEEE Workshop on Information Forensics and Security (WIFS) (2017), 1--6.

[3]

Jiawang Bai, Bin Chen, Yiming Li, Dongxian Wu, Weiwei Guo, Shu-Tao Xia, and En-Hui Yang. 2020. Targeted Attack for Deep Hashing based Retrieval. ArXiv, Vol. abs/2004.07955 (2020).

[4]

A. Brown, Weidi Xie, Vicky S. Kalogeiton, and Andrew Zisserman. 2020. Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval. ArXiv, Vol. abs/2007.12163 (2020).

[5]

Yue Cao, Mingsheng Long, Bin Liu, and Jianmin Wang. 2018. Deep Cauchy Hashing for Hamming Space Retrieval. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 1229--1237.

[6]

Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Philip S. Yu. 2017a. HashNet: Deep Learning to Hash by Continuation. 2017 IEEE International Conference on Computer Vision (ICCV) (2017), 5609--5618.

[7]

Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Philip S Yu. 2017b. HashNet: Deep Learning to Hash by Continuation. arXiv preprint arXiv:1702.00758 (2017).

[8]

Nicholas Carlini and David Wagner. 2017. Towards Evaluating the Robustness of Neural Networks. In IEEE Symp. Secur. Priv. IEEE, San Jose, CA, USA, 39--57. https://doi.org/10.1109/SP.2017.49

[9]

Tat-Seng Chua, J. Tang, R. Hong, Haojie Li, Zhiping Luo, and Y. Zheng. 2009. NUS-WIDE: a real-world web image database from National University of Singapore. In CIVR '09 .

[10]

B. Dai, R. Guo, S. Kumar, Niao He, and L. Song. 2017. Stochastic Generative Hashing. In ICML .

[11]

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In NIPS .

[12]

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6572

[13]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016a. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27--30, 2016 . IEEE Computer Society, 770--778. https://doi.org/10.1109/CVPR.2016.90

[14]

Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2016b. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 770--778.

[15]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6980

[16]

Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images.

[17]

Alex Krizhevsky and Geoffrey E. Hinton. 2011. Using very deep autoencoders for content-based image retrieval. In ESANN .

[18]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3--6, 2012, Lake Tahoe, Nevada, United States, Peter L. Bartlett, Fernando C. N. Pereira, Christopher J. C. Burges, Lé on Bottou, and Kilian Q. Weinberger (Eds.). 1106--1114. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks

Digital Library

[19]

Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2017. Adversarial examples in the physical world. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Workshop Track Proceedings. OpenReview.net. https://openreview.net/forum?id=HJGU3Rodl

[20]

Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 3270--3278.

[21]

Jie Li, Rongrong Ji, Hong Liu, Xiaopeng Hong, Yue Gao, and Qi Tian. 2019. Universal Perturbation Attack Against Image Retrieval. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 4898--4907. https://doi.org/10.1109/ICCV.2019.00500

[22]

Kevin Lin, Jiwen Lu, Chu-Song Chen, and J. Zhou. 2016. Learning Compact Binary Descriptors with Unsupervised Deep Neural Networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 1183--1192.

[23]

Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. ArXiv, Vol. abs/1405.0312 (2014).

[24]

Venice Erin Liong, Jiwen Lu, Gang Wang, Pierre Moulin, and Jie Zhou. 2015. Deep hashing for compact binary codes learning. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 2475--2483.

[25]

Haomiao Liu, R. Wang, S. Shan, and X. Chen. 2019 a. Deep Supervised Hashing for Fast Image Retrieval. International Journal of Computer Vision (2019), 1--18.

[26]

Zhuoran Liu, Zhengyu Zhao, and Martha Larson. 2019 b. Who's Afraid of Adversarial Queries?: The Impact of Image Modifications on Content-based Image Retrieval. Proceedings of the 2019 on International Conference on Multimedia Retrieval (2019).

Digital Library

[27]

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. 2016. DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27--30, 2016. IEEE Computer Society, 2574--2582. https://doi.org/10.1109/CVPR.2016.282

[28]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1409.1556

[29]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7--12, 2015 . IEEE Computer Society, 1--9. https://doi.org/10.1109/CVPR.2015.7298594

[30]

Giorgos Tolias, Filip Radenovic, and Ondrej Chum. 2019. Targeted Mismatch Adversarial Attack: Query With a Flower to Retrieve the Tower. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 5036--5045. https://doi.org/10.1109/ICCV.2019.00514

[31]

Dayan Wu, Zheng Lin, Bo Li, Mingzhen Ye, and Weiping Wang. 2017. Deep Supervised Hashing for Multi-Label and Large-Scale Image Retrieval. Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval (2017).

Digital Library

[32]

Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised Hashing for Image Retrieval via Image Representation Learning. In AAAI .

[33]

Erkun Yang, T. Liu, Cheng Deng, W. Liu, and Dacheng Tao. 2019. DistillHash: Unsupervised Deep Hashing by Distilling Data Pairs. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 2941--2950.

[34]

Erkun Yang, Tongliang Liu, Cheng Deng, and Dacheng Tao. 2020. Adversarial Examples for Hamming Space Search. IEEE Transactions on Cybernetics, Vol. 50 (2020), 1473--1484.

[35]

Andrew Zhai, Dmitry Kislyuk, Yushi Jing, Michael Feng, Eric Tzeng, Jeff Donahue, Yue Li Du, and Trevor Darrell. 2017. Visual Discovery at Pinterest. In WWW (Companion Volume) . ACM, 515--524.

[36]

Yanhao Zhang, Pan Pan, Yun Zheng, Kang Zhao, Yingya Zhang, Xiaofeng Ren, and Rong Jin. 2018. Visual Search at Alibaba. In KDD. ACM, 993--1001.

[37]

Fang Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. 2015. Deep semantic ranking based hashing for multi-label image retrieval. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 1556--1564.

[38]

Guoping Zhao, Mingyu Zhang, Jiajun Liu, and Ji-Rong Wen. 2019. Unsupervised Adversarial Attacks on Deep Feature-based Retrieval with GAN. ArXiv, Vol. abs/1907.05793 (2019).

[39]

Mo Zhou, Zhenxing Niu, Le Wang, Qilin Zhang, and Gang Hua. 2020. Adversarial Ranking Attack and Defense. CoRR, Vol. abs/2002.11293 (2020). arxiv: 2002.11293 https://arxiv.org/abs/2002.11293

[40]

Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. 2016. Deep Hashing Network for Efficient Similarity Retrieval. In AAAI .

Cited By

Meng FChen XCao YGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Targeted Universal Adversarial Attack on Deep Hash NetworksProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658062(165-174)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658062
Zhang ZZhang Z(2024)Semantic-Aware Adversarial TrainingBinary Representation Learning on Visual Images10.1007/978-981-97-2112-2_7(169-197)Online publication date: 7-Mar-2024
https://doi.org/10.1007/978-981-97-2112-2_7
Zhu FZhang WWu DWang LLi BWang W(2023)Targeted Transferable Attack against Deep Hashing RetrievalProceedings of the 5th ACM International Conference on Multimedia in Asia10.1145/3595916.3626420(1-7)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3595916.3626420
Show More Cited By

Index Terms

A Smart Adversarial Attack on Deep Hashing Based Image Retrieval
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
2. Security and privacy
  1. Software and application security

Recommendations

Targeted Attack and Defense for Deep Hashing
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Deep hashing methods have been intensively studied and successfully applied in massive fast image retrieval. However, inherited from the deficiency of deep neural networks, deep hashing models can be easily fooled by adversarial examples, which brings a ...
Efficient Query-based Black-box Attack against Cross-modal Hashing Retrieval
Deep cross-modal hashing retrieval models inherit the vulnerability of deep neural networks. They are vulnerable to adversarial attacks, especially for the form of subtle perturbations to the inputs. Although many adversarial attack methods have been ...
CgAT: Center-Guided Adversarial Training for Deep Hashing-Based Retrieval
WWW '23: Proceedings of the ACM Web Conference 2023

Deep hashing has been extensively utilized in massive image retrieval because of its efficiency and effectiveness. However, deep hashing models are vulnerable to adversarial examples, making it essential to develop adversarial defense methods for image ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '21: Proceedings of the 2021 International Conference on Multimedia Retrieval

August 2021

715 pages

ISBN:9781450384636

DOI:10.1145/3460426

General Chairs:
Wen-Huang Cheng
National Yang Ming Chiao Tung University, Taiwan
,
Mohan Kankanhalli
National University of Singapore, Singapore
,
Meng Wang
Hefei University of Technology, China
,
Program Chairs:
Wei-Ta Chu
National Cheng Kung University, Taiwan
,
Jiaying Liu
Peking University, China
,
Marcel Worring
University of Amsterdam, Netherlands

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2021

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article

Funding Sources

ARC Discovery Project
National Key Research and Development Program of China
Dongguan Social Science and Technology Development Key Project
Ten Thousand Talent Program
Liaoning Distinguished Professor
National Natural Science Foundation of China
Dongguan University of Technology

Conference

ICMR '21

Sponsor:

SIGMM

ICMR '21: International Conference on Multimedia Retrieval

August 21 - 24, 2021

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
766
Total Downloads

Downloads (Last 12 months)206
Downloads (Last 6 weeks)27

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Meng FChen XCao YGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Targeted Universal Adversarial Attack on Deep Hash NetworksProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658062(165-174)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658062
Zhang ZZhang Z(2024)Semantic-Aware Adversarial TrainingBinary Representation Learning on Visual Images10.1007/978-981-97-2112-2_7(169-197)Online publication date: 7-Mar-2024
https://doi.org/10.1007/978-981-97-2112-2_7
Zhu FZhang WWu DWang LLi BWang W(2023)Targeted Transferable Attack against Deep Hashing RetrievalProceedings of the 5th ACM International Conference on Multimedia in Asia10.1145/3595916.3626420(1-7)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3595916.3626420
Zhao WSong JYuan SGao LYang YShen HEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Precise Target-Oriented Attack against Deep Hashing-based RetrievalProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612364(6379-6389)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612364
Zhu LWang TLi JZhang ZShen JWang X(2023)Efficient Query-based Black-box Attack against Cross-modal Hashing RetrievalACM Transactions on Information Systems10.1145/355975841:3(1-25)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1145/3559758
Peng LQian JXu ZXin YGuo L(2023)Multi-Label Hashing for Dependency Relations Among Multiple ObjectivesIEEE Transactions on Image Processing10.1109/TIP.2023.325102832(1759-1773)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIP.2023.3251028
Yuan XZhang ZWang XWu L(2023)Semantic-Aware Adversarial Training for Reliable Deep Hashing RetrievalIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.329779118(4681-4694)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIFS.2023.3297791
Li YHao MLiu RZhang ZZhu HZhang Y(2023)Semantic-Aware Attack and Defense on Deep Hashing Networks for Remote-Sensing Image RetrievalIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.333679661(1-14)Online publication date: 2023
https://doi.org/10.1109/TGRS.2023.3336796
Han JLi PTao YRen P(2023)Encrypting Hashing Against LocalizationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.326678161(1-14)Online publication date: 2023
https://doi.org/10.1109/TGRS.2023.3266781
Lu JMiao YChen MHuang BLi BWang WVatsalan DKaafar M(2023)Distance Maximization and Defences on Deep Hashing Based Image Retrieval2023 IEEE International Conference on Knowledge Graph (ICKG)10.1109/ICKG59574.2023.00027(176-183)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICKG59574.2023.00027
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents