abstract

EmotionTracker: A Mobile Real-time Facial Expression Tracking System with the Assistant of Public AI-as-a-Service

Authors:

Xuanya LiAuthors Info & Claims

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Pages 4530 - 4532

https://doi.org/10.1145/3394171.3414447

Published: 12 October 2020 Publication History

Get Access

Abstract

Public AI-as-a-Service (AIaaS) is a promising next-generation computing paradigm that attracts resource-limited mobile users to outsource their machine learning tasks. However, the time delay between cloud/edge servers and end users makes it hard for real-time mobile artificial intelligence applications. In this demonstration, we present EmotionTracker, a real-time mobile facial expression tracking system combining AIaaS and mobile local auxiliary computing, including facial expression tracking and the corresponding task offloading. Mobile facial expression tracking iteratively estimates the facial expression with the help of sparse optical flow and neural network. Task offloading dynamically estimate the moment of task offloading with machine learning method. According to the results in a real-world environment, EmotionTracker successfully fulfills the mobile real-time facial expression tracking requirements.

Supplementary Material

MP4 File (3394171.3414447.mp4)

This video introduces the structure of the EmotionTracker and shows how the EmotionTracker works. EmotionTracker is a mobile real-time facial expression tracking system combining AIaaS and mobile local auxiliary computing. For EmotionTracker, the inner loop dominated by facial expression tracking and the outer loop dominated by task offloading make guarantee the real-time performance and the effectiveness of facial expression tracking respectively.

Download
10.07 MB

References

[1]

2020. liuxunchenglxc/EmotionTracker. https://github.com/liuxunchenglxc/EmotionTracker. Accessed May. 29, 2020.

Google Scholar

[2]

C. A. Corneanu, M. O. Simón, J. F. Cohn, and S. E. Guerrero. 2016. Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, 8 (Aug 2016), 1548--1568. https://doi.org/10.1109/TPAMI.2016.2515606

Crossref

Google Scholar

[3]

G. Hassan and K. Elgazzar. 2016. The case of face recognition on mobile devices. In 2016 IEEE Wireless Communications and Networking Conference. 1--6. https://doi.org/10.1109/WCNC.2016.7564975

Crossref

Google Scholar

[4]

M Shamim Hossain and Ghulam Muhammad. 2015. Cloud-assisted speech and face recognition framework for health monitoring. Mobile Networks and Applications, Vol. 20, 3 (2015), 391--399.

Digital Library

Google Scholar

[5]

Daniel McDuff, Abdelrahman Mahmoud, Mohammad Mavadati, May Amr, Jay Turcot, and Rana el Kaliouby. 2016. AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. In Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems. ACM, 3723--3726.

Digital Library

Google Scholar

[6]

Inchul Song, Hyun-Jun Kim, and Paul Barom Jeon. 2014. Deep learning for real-time robust facial expression recognition on a smartphone. In 2014 IEEE International Conference on Consumer Electronics (ICCE). IEEE, 564--567.

Crossref

Google Scholar

[7]

M. Suk and B. Prabhakaran. 2014. Real-Time Mobile Facial Expression Recognition System -- A Case Study. In 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops. 132--137. https://doi.org/10.1109/CVPRW.2014.25

Digital Library

Google Scholar

[8]

Huy Trinh, Prasad Calyam, Dmitrii Chemodanov, Shizeng Yao, Qing Lei, Fan Gao, and Kannappan Palaniappan. 2018. Energy-aware mobile edge computing and routing for low-latency visual data processing. IEEE Transactions on Multimedia, Vol. 20, 10 (2018), 2562--2577.

Crossref

Google Scholar

Cited By

View all

Condegni AWang WLi R(2023)A Digital Human System with Realistic Facial Expressions for Friendly Human-Machine InteractionAdvanced Intelligent Computing Technology and Applications10.1007/978-981-99-4755-3_68(787-798)Online publication date: 30-Jul-2023
https://doi.org/10.1007/978-981-99-4755-3_68

Index Terms

EmotionTracker: A Mobile Real-time Facial Expression Tracking System with the Assistant of Public AI-as-a-Service

Recommendations

A Video-Based Facial Motion Tracking and Expression Recognition System

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of ...
Modelling Task Offloading Mobile Edge Computing
ICCDE '22: Proceedings of the 2022 8th International Conference on Computing and Data Engineering

With the rapid growth of mobile devices (such as smart phones and IoT devices) and the upcoming 5G era, it has been considered that edge computing will play a significant role, which together with the Cloud server forms the Mobile Edge Computing (MEC) ...
Real-Time Mobile Facial Expression Recognition System -- A Case Study
CVPRW '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops

This paper presents a mobile application for real time facial expression recognition running on a smart phone with a camera. The proposed system uses a set of Support Vector Machines (SVMs) for classifying 6 basic emotions and neutral expression along ...

Comments

Information & Contributors

Information

Published In

MM '20: Proceedings of the 28th ACM International Conference on Multimedia

October 2020

4889 pages

ISBN:9781450379885

DOI:10.1145/3394171

General Chairs:
Chang Wen Chen
Chinese University of Hong Kong, Shenzhen, China
,
Rita Cucchiara
UNIMORE, Italy
,
Xian-Sheng Hua
Alibaba Group, China
,
Program Chairs:
Guo-Jun Qi
Futurewei Technologies, USA
,
Elisa Ricci
UNITN & Fondazione Bruno Kessler, Italy
,
Zhengyou Zhang
Tencent, China
,
Roger Zimmermann
National University of Singapore, Singapore

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

National Key Research and Development Program of China
the NSFC

Conference

MM '20

Sponsor:

SIGMM

MM '20: The 28th ACM International Conference on Multimedia

October 12 - 16, 2020

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
154
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Condegni AWang WLi R(2023)A Digital Human System with Realistic Facial Expressions for Friendly Human-Machine InteractionAdvanced Intelligent Computing Technology and Applications10.1007/978-981-99-4755-3_68(787-798)Online publication date: 30-Jul-2023
https://doi.org/10.1007/978-981-99-4755-3_68

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A Video-Based Facial Motion Tracking and Expression Recognition System

Modelling Task Offloading Mobile Edge Computing

Real-Time Mobile Facial Expression Recognition System -- A Case Study