Work in Progress

FakeBuster: A DeepFakes Detection Tool for Video Conferencing Scenarios

Authors:

Ramanathan Subramanian,

Abhinav DhallAuthors Info & Claims

IUI '21 Companion: Companion Proceedings of the 26th International Conference on Intelligent User Interfaces

Pages 61 - 63

https://doi.org/10.1145/3397482.3450726

Published: 14 April 2021 Publication History

Abstract

This paper proposes FakeBuster, a novel DeepFake detector for (a) detecting impostors during video conferencing, and (b) manipulated faces on social media. FakeBuster is a standalone deep learning- based solution, which enables a user to detect if another person’s video is manipulated or spoofed during a video conference-based meeting. This tool is independent of video conferencing solutions and has been tested with Zoom and Skype applications. It employs a 3D convolutional neural network for predicting video fakeness. The network is trained on a combination of datasets such as Deeperforensics, DFDC, VoxCeleb, and deepfake videos created using locally captured images (specific to video conferencing scenarios). Diversity in the training data makes FakeBuster robust to multiple environments and facial manipulations, thereby making it generalizable and ecologically valid.

References

[1]

1995. QT- One framework. One codebase. Any platform.https://www.qt.io/

[2]

2016. PyQT- Python bindings for The Qt Company’s Qt application framework. https://riverbankcomputing.com/software/pyqt/intro

[3]

2020. Python MSS- An ultra fast cross-platform multiple screenshots module in pure python using ctypes.https://github.com/BoboTiG/python-mss

[4]

2020. Reality Defender 2020: A FORCE AGAINST DEEPFAKES. https://rd2020.org/index.html

[5]

Ali Aliev. 2019. Avatarify- Photorealistic avatars for video-conferencing apps.https://github.com/alievk/avatarify

[6]

G. Bradski. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools(2000).

[7]

Tom Bur. 2020. New Steps to Combat Disinformation. https://blogs.microsoft.com/on-the-issues/2020/09/01/disinformation-deepfakes-newsguard-video-authenticator/

[8]

Komal Chugh, Parul Gupta, Abhinav Dhall, and Ramanathan Subramanian. 2020. Not Made for Each Other- Audio-Visual Dissonance-Based Deepfake Detection and Localization. In Proceedings of the 28th ACM International Conference on Multimedia (Seattle, WA, USA) (MM ’20). Association for Computing Machinery, New York, NY, USA, 439–447. https://doi.org/10.1145/3394171.3413700

Digital Library

[9]

Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram, and Cristian Canton Ferrer. 2019. The Deepfake Detection Challenge (DFDC) Preview Dataset. arxiv:1910.08854 [cs.CV]

[10]

Parul Gupta, Komal Chugh, Abhinav Dhall, and Ramanathan Subramanian. 2020. The Eyes Know It: FakeET- An Eye-Tracking Database to Understand Deepfake Perception. In Proceedings of the 2020 International Conference on Multimodal Interaction (Virtual Event, Netherlands) (ICMI ’20). Association for Computing Machinery, New York, NY, USA, 519–527. https://doi.org/10.1145/3382507.3418857

Digital Library

[11]

Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. 2017. Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?CoRR abs/1711.09577(2017). arxiv:1711.09577http://arxiv.org/abs/1711.09577

[12]

Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy. 2020. DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection. arxiv:2001.03024 [cs.CV]

[13]

Tero Karras, Samuli Laine, and Timo Aila. 2019. A Style-Based Generator Architecture for Generative Adversarial Networks. arxiv:1812.04948 [cs.NE]

[14]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).

[15]

Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. VoxCeleb: A Large-Scale Speaker Identification Dataset. Interspeech 2017 (Aug 2017). https://doi.org/10.21437/interspeech.2017-950

[16]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf

Digital Library

[17]

Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. 2019. First order motion model for image animation. In Advances in Neural Information Processing Systems. 7137–7147.

Cited By

Akhtar ZPendyala TAthmakuri V(2024)Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the CurveForensic Sciences10.3390/forensicsci40300214:3(289-377)Online publication date: 13-Jul-2024
https://doi.org/10.3390/forensicsci4030021
Al-Dulaimi OKurnaz S(2024)A Hybrid CNN-LSTM Approach for Precision Deepfake Image Detection Based on Transfer LearningElectronics10.3390/electronics1309166213:9(1662)Online publication date: 25-Apr-2024
https://doi.org/10.3390/electronics13091662
Li RYin HLi YLi H(2024)V3CViT: Deepfake Detection Based on Video Vision Transformer and 3D Convolution NetworkProceedings of 2024 Chinese Intelligent Systems Conference10.1007/978-981-97-8654-1_32(307-316)Online publication date: 27-Oct-2024
https://doi.org/10.1007/978-981-97-8654-1_32
Show More Cited By

Index Terms

FakeBuster: A DeepFakes Detection Tool for Video Conferencing Scenarios

Index terms have been assigned to the content through auto-classification.

Recommendations

A Method for Deepfake Detection Using Convolutional Neural Networks
Abstract—
This paper proposes a method of countering spoofing attacks by improving the resilience of face-based biometric authentication systems to digital face manipulation attacks on the biometric input module. The proposed method of digital face ...
Biometric person authentication with liveness detection based on audio-visual fusion

In this paper, we propose two new approaches for extracting mouth features for authenticating the person identity with liveness checks. The novel correlated audio-lip features and tensor lip-motion features allow liveness checks to be included in the ...
Detecting liveness of fingerprint biometrics

Biometrics refer to automated recognition of individuals based on their biological and behavioral characteristics. Biometric systems are widely used for security. But biometric systems are vulnerable to a certain type of attack. The type 1 attack or ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '21 Companion: Companion Proceedings of the 26th International Conference on Intelligent User Interfaces

April 2021

101 pages

ISBN:9781450380188

DOI:10.1145/3397482

Co-chairs:
Katrien Verbert
Katholieke Universiteit Leuven
,
Dennis Parra
Pontificia Universidad Católica de Chile
,
General Chair:
Tracy Hammond
Texas A&M University
,
Program Chairs:
Bart Knijnenburg
Clemson University
,
John O'Donovan
UC Santa Barbara
,
Paul Taele
Texas A&M University

Copyright © 2021 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 April 2021

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Conference

IUI '21

Sponsor:

IUI '21: 26th International Conference on Intelligent User Interfaces

April 14 - 17, 2021

TX, College Station, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
345
Total Downloads

Downloads (Last 12 months)70
Downloads (Last 6 weeks)2

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Akhtar ZPendyala TAthmakuri V(2024)Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the CurveForensic Sciences10.3390/forensicsci40300214:3(289-377)Online publication date: 13-Jul-2024
https://doi.org/10.3390/forensicsci4030021
Al-Dulaimi OKurnaz S(2024)A Hybrid CNN-LSTM Approach for Precision Deepfake Image Detection Based on Transfer LearningElectronics10.3390/electronics1309166213:9(1662)Online publication date: 25-Apr-2024
https://doi.org/10.3390/electronics13091662
Li RYin HLi YLi H(2024)V3CViT: Deepfake Detection Based on Video Vision Transformer and 3D Convolution NetworkProceedings of 2024 Chinese Intelligent Systems Conference10.1007/978-981-97-8654-1_32(307-316)Online publication date: 27-Oct-2024
https://doi.org/10.1007/978-981-97-8654-1_32
Nagothu DPoredi NChen Y(2023)Evolution of Attacks on Intelligent Surveillance Systems and Effective Detection TechniquesIntelligent Video Surveillance - New Perspectives10.5772/intechopen.105958Online publication date: 8-Feb-2023
https://doi.org/10.5772/intechopen.105958
Akhtar Z(2023)Deepfakes Generation and Detection: A Short SurveyJournal of Imaging10.3390/jimaging90100189:1(18)Online publication date: 13-Jan-2023
https://doi.org/10.3390/jimaging9010018
Jung JKang CYoon JWoo SHan JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)SAFE: Sequential Attentive Face Embedding with Contrastive Learning for Deepfake Video DetectionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615279(3993-3997)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615279
Panda PNicholas MNguyen DOfek EPahud MRintel SGonzalez-Franco MHinckley KLanier J(2023)Beyond Audio: Towards a Design Space of Headphones as a Site for Interaction and SensingProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596022(904-916)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596022
Neha Arora B(2023)Deep Learning based Model for Deepfake Image Detection: An Analytical Approach2023 3rd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA)10.1109/ICIMIA60377.2023.10426561(1019-1027)Online publication date: 21-Dec-2023
https://doi.org/10.1109/ICIMIA60377.2023.10426561
Nawaz MJaved AIrtaza A(2023)Convolutional long short-term memory-based approach for deepfakes detection from videosMultimedia Tools and Applications10.1007/s11042-023-16196-x83:6(16977-17000)Online publication date: 19-Jul-2023
https://doi.org/10.1007/s11042-023-16196-x
Sandotra NArora B(2023)A comprehensive evaluation of feature-based AI techniques for deepfake detectionNeural Computing and Applications10.1007/s00521-023-09288-036:8(3859-3887)Online publication date: 14-Dec-2023
https://dl.acm.org/doi/10.1007/s00521-023-09288-0
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten