research-article

Fuzzy triangulation signature for detection of change in human emotion from face video image sequence

Authors:

Paramartha Dutta,

Avishek NandiAuthors Info & Claims

Multimedia Tools and Applications, Volume 80, Issue 21-23

Pages 31993 - 32022

https://doi.org/10.1007/s11042-021-11196-1

Published: 01 September 2021 Publication History

Abstract

The present article proposes a geometry-based fuzzy relational technique for capturing gradual change in human emotion over time available from relevant face image sequences. As associated features, we make use of fuzzy membership arising out of five triangle signatures such as - (i) Fuzzy Isosceles Triangle Signature (FIS), (ii) Fuzzy Right Triangle Signature (FRS), (iii) Fuzzy Right Isosceles Triangle Signature (FIRS), (iv) Fuzzy Equilateral Triangle Signature (FES), and (v) Other Fuzzy Triangles Signature (OFS) to achieve the task of appropriate classification of facial transition from neutrality to one among the six expressions viz. anger (AN), disgust (DI), fear (FE), happiness (HA), sadness (SA) and surprise (SU). The effectiveness of the Multilayer Perceptron (MLP) classifier is tested and validated through 10 fold cross-validation method on three benchmark image sequence datasets namely Extended Cohn-Kanade (CK+), M&M Initiative (MMI), and Multimedia Understanding Group (MUG). Experimental outcomes are found to have achieved accuracy to the tune of 98.47%, 93.56%, and 99.25% on CK+, MMI, and MUG respectively vindicating the effectiveness by exhibiting the superiority of our proposed technique in comparison to other state-of-the-art methods in this regard.

References

[1]

Ahn B, Han Y, Kweon IS (2012) Real-time facial landmarks tracking using active shape model and lk optical flow. In: 2012 9th international conference on ubiquitous robots and ambient intelligence (URAI). IEEE, pp 541–543

[2]

Aifanti N, Papachristou C, Delopoulos A (2010) The mug facial expression database. In: 11Th international workshop on image analysis for multimedia interactive services WIAMIS 10. IEEE, pp 1–4

[3]

Barman A Human Emotion Recognition from Face Images 2020 Berlin Springer

[4]

Barman A, Dutta P (2017) Facial expression recognition using distance and shape signature features. Pattern Recognit Lett

[5]

Barman A and Dutta P Facial expression recognition using distance and texture signature relevant features Appl Soft Comput 2019 77 88-105

[6]

Barman A and Dutta P Influence of shape and texture features on facial expression recognition IET Image Process 2019 13 8 1349-1363

[7]

Bastanfard A, Bastanfard O, Takahashi H, and Nakajima M Toward anthropometrics simulation of face rejuvenation and skin cosmetic Comput Animat Virt Worlds 2004 15 3-4 347-352

[8]

Bastanfard A, Takahashi H, Nakajima M (2004) Toward e-appearance of human face and hair by age, expression and rejuvenation. In: 2004 International conference on cyberworlds. IEEE, pp 306– 311

[9]

Choi H-C, Oh S-Y (2006) Realtime facial expression recognition using active appearance model and multilayer perceptron. In: 2006 SICE-ICASE International joint conference. IEEE, pp 5924–5927

[10]

Cootes TF, Taylor JC, Cooper DH, and Graham J Active shape models-their training and application Comput Vision Image Understand 1995 61 1 38-59

[11]

Ekman P, Friesen WV (2003) Unmasking the face: A guide to recognizing emotions from facial clues. Ishk

[12]

Ghimire D and Lee J Extreme learning machine ensemble using bagging for facial expression recognition JIPS 2014 10 3 443-458

[13]

Ghimire D, Lee J, Li Z-N, and Jeong S Recognition of facial expressions based on salient geometric features and support vector machines Multimed Tools Appl 2017 76 6 7921-7946

[14]

Gross R, Matthews I, and Baker S Appearance-based face recognition and light-fields IEEE Trans Pattern Anal Mach Intell 2004 26 4 449-465

[15]

Happy SL, Routray A (2015) Robust facial expression classification using shape and appearance features. In: 2015 Eighth international conference on advances in pattern recognition (ICAPR). IEEE, pp 1–5

[16]

Kotsia I and Pitas I Facial expression recognition in image sequences using geometric deformation features and support vector machines IEEE Trans Image Process 2006 16 1 172-187

[17]

Kumari J, Rajesh R, and Pooja KM Facial expression recognition: A survey Procedia Comput Sci 2015 58 1 486-491

[18]

Li Y, Wang S, Zhao Y, and Ji Q Simultaneous facial feature tracking and facial expression recognition IEEE Trans Image Process 2013 22 7 2559-2573

[19]

Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 Ieee computer society conference on computer vision and pattern recognition-workshops. IEEE, pp 94–101

[20]

Majumder A, Behera L, and Subramanian VK Emotion recognition from geometric facial features using self-organizing map Pattern Recogn 2014 47 3 1282-1293

[21]

Meftah IT, Thanh NL, Amar CB (2012) Emotion recognition using knn classification for user modeling and sharing of affect states. In: International conference on neural information processing. Springer, pp 234–242

[22]

Mehrabian A and Russell JA An approach to environmental psychology 1974 Cambridge MIT Press

[23]

Mohammadian A, Aghaeinia H, and Towhidkhah F Video-based facial expression recognition by removing the style variations IET Image Process 2015 9 7 596-603

[24]

Ojala T, Pietikäinen M, and Harwood D A comparative study of texture measures with classification based on featured distributions Pattern Recognit 1996 29 1 51-59

[25]

Pantic M, Pentland A, Nijholt A, Huang TS (2007) Machine computing and understanding of human behavior Human A survey. In: Artifical intelligence for human computing. Springer, pp 47–71

[26]

Perikos I, Ziakopoulos E, Hatzilygeroudis I (2014) Recognizing emotions from facial expressions using neural network. In: IFIP International conference on artificial intelligence applications and innovations. Springer, pp 236–245

[27]

Rahul M, Kohli N, Agarwal R, and Mishra S Facial expression recognition using geometric features and modified hidden markov model Int J Grid Util Comput 2019 10 5 488-496

[28]

Rahulamathavan Y, Phan RC-W, Chambers JA, and Parish DJ Facial expression recognition in the encrypted domain based on local fisher discriminant analysis IEEE Trans Affect Comput 2012 4 1 83-92

[29]

Saeed A, Al-Hamadi A, Niese R, Elzobi M (2014) Frame-based facial expression recognition using geometrical features. Adv Human-Comput Interact 2014

[30]

Samadiani N, Huang G, Cai B, Luo W, Chi C-H, Xiang Y, and He J A review on automatic facial expression recognition systems assisted by multimodal sensor data Sensors 2019 19 8 1863

[31]

Shan C, Gong S, and McOwan PW Facial expression recognition based on local binary patterns: A comprehensive study Image Vision Comput 2009 27 6 803-816

[32]

Sharma G, Singh L, and Gautam S Automatic facial expression recognition using combined geometric features 3D Res 2019 10 2 14

[33]

Terzopoulos D and Waters K Analysis and synthesis of facial image sequences using physical and anatomical models IEEE Trans Pattern Anal Mach Intell 1993 15 6 569-579

[34]

Tian Y-L, Kanade T, Cohn JF (2005) Facial expression analysis. In: Handbook of face recognition. Springer, pp 247–275

[35]

Tzimiropoulos G, Pantic M (2013) Optimization problems for fast aam fitting in-the-wild. In: Proceedings of the IEEE international conference on computer vision, pp 593–600

[36]

Valstar M, Pantic M (2010) Induced disgust, happiness and surprise: an addition to the mmi facial expression database. In: Proc. 3rd intern. Workshop on EMOTION (satellite of LREC): Corpora for research on emotion and affect, Paris, France, p 65

[37]

Vishnu Priya R Emotion recognition from geometric fuzzy membership functions Multimed Tools Appl 2019 78 13 17847-17878

[38]

Wolf L (2009) Face recognition, geometric vs. appearance-based. Encycloped Biomet 2

[39]

Yaddaden Y, Adda M, Bouzouane A, Gaboury S, Bouchard B (2017) Facial expression recognition from video using geometric features

[40]

Yeasin M, Bullot B, and Sharma R Recognition of facial expressions and measurement of levels of interest from video IEEE Trans Multimed 2006 8 3 500-508

[41]

Zangeneh E and Moradi A Facial expression recognition by using differential geometric features The Imaging Sci J 2018 66 8 463-470

[42]

Zhang S, Pan X, Cui Y, Zhao X, and Liu L Learning affective video features for facial expression recognition via hybrid deep learning IEEE Access 2019 7 32297-32304

[43]

Zhang S, Zhao X, and Lei B Robust facial expression recognition via compressive sensing Sensors 2012 12 3 3747-3761

[44]

Zhao L, Wang Z, Zhang G (2017) Facial expression recognition from video sequences based on spatial-temporal motion local binary pattern and gabor multiorientation fusion histogram. Math Probl Eng 2017

Cited By

Senthil Sivakumar MGurumekala TMegalan Leo LThandaiah Prabu R(2023)Expert System for Smart Virtual Facial Emotion Detection Using Convolutional Neural NetworkWireless Personal Communications: An International Journal10.1007/s11277-024-10867-0133:4(2297-2319)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s11277-024-10867-0
Kumar KReddy H(2023)An optimized whale-based modular neural framework to predict crime eventsMultimedia Tools and Applications10.1007/s11042-023-14660-282:20(30855-30873)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1007/s11042-023-14660-2

Index Terms

Fuzzy triangulation signature for detection of change in human emotion from face video image sequence
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
    2. Knowledge representation and reasoning
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Index terms have been assigned to the content through auto-classification.

Recommendations

Landmark triangulation-induced Altitude Signature for change detection of human emotion from face image sequence
Abstract
Video-based facial expression recognition is a potential alternative for detecting the transitional processes of human emotion. In this paper, we present the landmark triangulation-induced altitude signatures to depict gradual changes in human ...
Recognition of human emotion transition from video sequence using triangulation induced various centre pairs distance signatures
Abstract
Human emotion recognition manifests the concept of contextual understanding of human communication in nonverbal form. This fascinates us to monitor the dynamic information of an emotion. In this article, we present an automatic recognition system ...
Highlights
- Presenting a face sequence-based method for automatic emotional signals recognition.
- Studying the significance of facial feature points on face video frames.
- Introducing the triangulation method to explore six center pairs distance ...
Face/Non-face Classification Method Based on Partial Face Classifier Using LDA and MLP
ICIS '10: Proceedings of the 2010 IEEE/ACIS 9th International Conference on Computer and Information Science

In this paper, we proposed a face / non-face classification method based on a partial face classifier using LDA and MLP for face detection system. General classifier for face and non-face classification requires many face images and background images ...

Comments

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 80, Issue 21-23

Sep 2021

1592 pages

ISSN:1380-7501

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 September 2021

Accepted: 28 June 2021

Revision received: 16 January 2021

Received: 28 October 2020

Author Tags

Qualifiers

Research-article

Funding Sources

Department of Science and Technology, Ministry of Science and Technology

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Senthil Sivakumar MGurumekala TMegalan Leo LThandaiah Prabu R(2023)Expert System for Smart Virtual Facial Emotion Detection Using Convolutional Neural NetworkWireless Personal Communications: An International Journal10.1007/s11277-024-10867-0133:4(2297-2319)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s11277-024-10867-0
Kumar KReddy H(2023)An optimized whale-based modular neural framework to predict crime eventsMultimedia Tools and Applications10.1007/s11042-023-14660-282:20(30855-30873)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1007/s11042-023-14660-2

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents