EfficientNet-based multi-dimensional network optimization for Deepfake video detection
Authors: 
Yong Zhang, Xinqi Zhang, Bingjie LiAuthors Info & Claims
Yong Zhang, Xinqi Zhang, Bingjie LiAuthors Info & ClaimsPages 433 - 438
Abstract
The rapid development of deep learning techniques, especially generative adversarial networks, has led to the generation of very realistic forged faces. Deepfake technology has brought convenience to society and also produced a large number of undesirable effects. Many detectors have been developed to defend videos generated by Deepfake manipulation techniques. In this paper, we take low overhead and high performance as the task of Deepfake detection and propose a new EfficientNet-B0 combined with ViT Deepfake detection network, which consists of two key components: (1) ConvFFN block, which brings a larger receptive field to the network; (2) Transformer Encoder with Hilo, which separates the high and low frequencies of the attention layer to focus on the global structure. We conducted extensive experiments on the DFDC dataset and the FF++ dataset to demonstrate the stability and practical applicability of the proposed method.
References
[1]
Korshunov, Pavel, and Sébastien Marcel. "Deepfakes: a new threat to face recognition? assessment and detection." arXiv preprint arXiv:1812.08685 (2018).
[2]
Neves, Joao C., "Ganprintr: Improved fakes and evaluation of the state of the art in face manipulation detection." IEEE Journal of Selected Topics in Signal Processing 14.5 (2020): 1038-1048.
[3]
Dang, Hao, "On the detection of digital face manipulation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition. 2020.
[4]
Wang, Zhi, Yiwen Guo, and Wangmeng Zuo. "Deepfake forensics via an adversarial game." IEEE Transactions on Image Processing 31 (2022): 3541-3552.
[5]
Deepfake Detection Challenge. https://deepfakedetectionchallenge.ai/
[6]
FaceApp. https://apps.apple.com/gb/app/faceapp-ai-face-editor/id1180884341
[7]
ZAO. https://apps.apple.com/cn/app/id1465199127
[8]
Goodfellow, Ian, "Generative adversarial networks." Communications of the ACM 63.11 (2020): 139-144.
[9]
FaceSwap. https://github.com/ MarekKowalski/FaceSwap/. Accessed: 2018-10-29.
[10]
Thies, Justus, Michael Zollhöfer, and Matthias Nießner. "Deferred neural rendering: Image synthesis using neural textures." Acm Transactions on Graphics (TOG) 38.4 (2019): 1-12.
[11]
Thies, Justus, "Face2face: Real-time face capture and reenactment of rgb videos." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[12]
Xu, Zhiliang, "StyleSwap: Style-Based Generator Empowers Robust Face Swapping." Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIV. Cham: Springer Nature Switzerland, 2022.
[13]
Shu, Changyong, "Few-shot head swapping in the wild." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[14]
Xu, Yangyang, "High-resolution face swapping via latent semantics disentanglement." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[15]
Xu, Zhiliang, "Mobilefaceswap: A lightweight framework for video face swapping." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 3. 2022.
[16]
Boháček, Matyáš, and Hany Farid. "Protecting President Zelenskyy Against Deep Fakes." arXiv preprint arXiv:2206.12043 (2022).
[17]
Khalil, Hady A., and Shady A. Maged. "Deepfakes creation and detection using deep learning." 2021 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC). IEEE, 2021.
[18]
Cozzolino, Davide, Matthias Nießner, and Luisa Verdoliva. "Audio-visual person-of-interest deepfake detection." arXiv preprint arXiv:2204.03083 (2022).
[19]
Chen, Beijing, Tianmu Li, and Weiping Ding. "Detecting deepfake videos based on spatiotemporal attention and convolutional LSTM." Information Sciences 601 (2022): 58-70.
[20]
Heo, Young-Jin, Woon-Ha Yeo, and Byung-Gyu Kim. "Deepfake detection algorithm based on improved vision transformer." Applied Intelligence 53.7 (2022): 7512-7527.
[21]
Zhao, Hanqing, "Multi-attentional deepfake detection." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.
[22]
Yang, Xin, Yuezun Li, and Siwei Lyu. "Exposing deep fakes using inconsistent head poses." ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019.
[23]
Li, Yuezun, Ming-Ching Chang, and Siwei Lyu. "In ictu oculi: Exposing ai created fake videos by detecting eye blinking." 2018 IEEE international workshop on information forensics and security (WIFS). IEEE, 2018.
[24]
Rossler, Andreas, "Faceforensics++: Learning to detect manipulated facial images." Proceedings of the IEEE/CVF international conference on computer vision. 2019.
[25]
Dong, Xiaoyi, "Protecting celebrities from deepfake with identity consistency transformer." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[26]
Lu, Changlei, "Deepfake video detection using 3D-attentional inception convolutional neural network." 2021 IEEE International Conference on Image Processing (ICIP). IEEE, 2021.
[27]
Dosovitskiy, Alexey, "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020).
[28]
Radford, Alec, "Improving language understanding by generative pre-training." (2018).
[29]
Radford, Alec, "Language models are unsupervised multitask learners." OpenAI blog 1.8 (2019): 9.
[30]
Dong, Xiaoyi, "Cswin transformer: A general vision transformer backbone with cross-shaped windows." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[31]
Liu, Ze, "Swin transformer v2: Scaling up capacity and resolution." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.
[32]
Zamir, Syed Waqas, "Restormer: Efficient transformer for high-resolution image restoration." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[33]
Wang, Rui, "Bevt: Bert pretraining of video transformers." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[34]
Dong, Xiaoyi, "Protecting celebrities from deepfake with identity consistency transformer." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.
[35]
Wodajo, Deressa, and Solomon Atnafu. "Deepfake video detection using convolutional vision transformer." arXiv preprint arXiv:2102.11126 (2021).
[36]
Deng, Liwei, Jiandong Wang, and Zhen Liu. "Cascaded Network Based on EfficientNet and Transformer for Deepfake Video Detection." Neural Processing Letters (2023): 1-20.
[37]
Seferbekov S (2020) https://github.com/selimsef/dfdc_deepfake_challenge. Accessed 24 Jan 2022
[38]
Coccomini, Davide Alessandro, "Combining efficientnet and vision transformers for video deepfake detection." Image Analysis and Processing–ICIAP 2022: 21st International Conference, Lecce, Italy, May 23–27, 2022, Proceedings, Part III. Cham: Springer International Publishing, 2022.
Recommendations
Deepfake generation and detection, a survey
AbstractDeepfake refers to realistic, but fake images, sounds, and videos generated by articial intelligence methods. Recent advances in deepfake generation make deepfake more realistic and easier to make. Deepfake has been a signicant threat to national ...
Autoencoder-based Data Augmentation for Deepfake Detection
MAD '23: Proceedings of the 2nd ACM International Workshop on Multimedia AI against DisinformationImage generation has seen huge leaps in the last few years. Less than 10 years ago we could not generate accurate images using deep learning at all, and now it is almost impossible for the average person to distinguish a real image from a generated one. ...
Adversarial Attack on Deepfake Detection Using RL Based Texture Patches
Computer Vision – ECCV 2020 WorkshopsAbstractThe advancements in GANs have made creating deepfake videos a relatively easy task. Considering the threat that deepfake videos pose for manipulating political opinion, recent research has focused on ways to better detect deepfake videos. Even ...
Comments
Information & Contributors
Information
Published In
September 2023
1540 pages
ISBN:9798400707674
DOI:10.1145/3641584
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
AIPR 2023
AIPR 2023: 2023 6th International Conference on Artificial Intelligence and Pattern Recognition
September 22 - 24, 2023
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 22Total Downloads
- Downloads (Last 12 months)22
- Downloads (Last 6 weeks)7
Reflects downloads up to 28 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format