research-article

Improving Subject-independent Human Emotion Recognition Using Electrodermal Activity Sensors for Active and Assisted Living

Authors:

Fadi Al Machot,

Suneth Ranasinghe,

Ahmad Haj Mosa,

Kyamakya KyandoghereAuthors Info & Claims

PETRA '18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

Pages 222 - 228

https://doi.org/10.1145/3197768.3201523

Published: 26 June 2018 Publication History

Abstract

In Active and Assisted Living environments (AAL), one of the major tasks is to make sure that old people or disabled persons do feel well in their environment. Unfortunately, it is still a difficult task to design a learning system or build a machine learning model which can be trained on a group of subjects using physiological sensors and performs well when testing it on other subjects. This paper proposes a dynamic calibration algorithm which presents promising results for subject-independent human emotion recognition. The goal of the calibration module is to calibrate itself with respect to the features of a new subject by finding the most similar subject in the training data. In order to check the overall performance, this approach is tested using the well-known MAHNOB dataset. The results show a promising improvement based on different evaluation metrics, e.g., sensitivity and specificity.

References

[1]

Per Ahlgren, Bo Jarneving, and Ronald Rousseau. 2003. Requirements for a cocitation similarity measure, with special reference to Pearson's correlation coefficient. Journal of the Association for Information Science and Technology 54, 6 (2003), 550--560.

Digital Library

[2]

Fadi Al Machot, Ahmad Haj Mosa, Alireza Fasih, Christopher Schwarzlmüller, Mouhanndad Ali, and Kyandoghere Kyamakya. 2012. A novel real-time emotion detection system for advanced driver assistance systems. In Autonomous Systems: Developments and Trends. Springer, 267--276.

[3]

Naomi S Altman. 1992. An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician 46, 3 (1992), 175--185.

[4]

David Arthur and Sergei Vassilvitskii. 2007. k-means++: The advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms. Society for Industrial and Applied Mathematics, 1027--1035.

Digital Library

[5]

Mathias Benedek and Christian Kaernbach. 2010. Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology 47, 4 (2010), 647--658.

[6]

Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry 25, 1 (1994), 49--59.

[7]

Roddy Cowie, Ellen Douglas-Cowie, Nicolas Tsapatsoulis, George Votsis, Stefanos Kollias, Winfried Fellenz, and John G Taylor. 2001. Emotion recognition in human-computer interaction. IEEE Signal processing magazine 18, 1 (2001), 32--80.

[8]

Irfan A Essa and Alex Pentland. 1994. A vision system for observing and extracting facial action parameters. In CVPR. 76--83.

[9]

Nico H Frijda. 1986. The emotions Cambridge University Press Cambridge. UK Google Scholar (1986).

[10]

Keinosuke Fukunaga. 2013. Introduction to statistical pattern recognition. Academic press.

[11]

Alberto Greco, Antonio Lanata, Luca Citi, Nicola Vanello, Gaetano Valenza, and Enzo Pasquale Scilingo. 2016. Skin admittance measurement for emotion recognition: A study over frequency sweep. Electronics 5, 3 (2016), 46.

[12]

Greg Hamerly and Charles Elkan. 2004. Learning the k in k-means. In Advances in neural information processing systems. 281--288.

Digital Library

[13]

Cheng He, Yun-jin Yao, and Xue-song Ye. 2017. An Emotion Recognition System Based on Physiological Signals Obtained by Wearable Sensors. In Wearable Sensors and Robots. Springer, 15--25.

[14]

Suwicha Jirayucharoensak, Setha Pan-Ngum, and Pasin Israsena. 2014. EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. The Scientific World Journal 2014 (2014).

[15]

Ian Jolliffe. 2002. Principal component analysis. Wiley Online Library.

[16]

Samira Ebrahimi Kahou, Christopher Pal, Xavier Bouthillier, Pierre Froumenty, Çaglar Gülçehre, Roland Memisevic, Pascal Vincent, Aaron Courville, Yoshua Bengio, Raul Chandias Ferrari, et al. 2013. Combining modality specific deep neural networks for emotion recognition in video. In Proceedings of the 15th ACM on International conference on multimodal interaction. ACM, 543--550.

Digital Library

[17]

Cornelia Kappeler-Setz, Franz Gravenhorst, Johannes Schumm, Bert Arnrich, and Gerhard Tröster. 2013. Towards long term monitoring of electrodermal activity in daily life. Personal and ubiquitous computing 17, 2 (2013), 261--271.

Digital Library

[18]

Gil Keren, Tobias Kirschstein, Erik Marchi, Fabien Ringeval, and Björn Schuller. {n. d.}. END-TO-END Learning for Dimensional Emotion Recognition from Physiological Signals. ({n. d.}).

[19]

Mahdi Khezri, Mohammad Firoozabadi, and Ahmad Reza Sharafat. 2015. Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals. Computer methods and programs in biomedicine 122, 2 (2015), 149--164.

Digital Library

[20]

Gil Levi and Tal Hassner. 2015. Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, 503--510.

Digital Library

[21]

Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing 7, 1 (2003), 76--80.

Digital Library

[22]

Anima Majumder, Laxmidhar Behera, and Venkatesh K Subramanian. 2014. Emotion recognition from geometric facial features using self-organizing map. Pattern Recognition 47, 3 (2014), 1282--1293.

Digital Library

[23]

MATLAB. 2015. version R2015a. (2015).

[24]

Hongying Meng, Di Huang, Heng Wang, Hongyu Yang, Mohammed AI-Shuraifi, and Yunhong Wang. 2013. Depression recognition based on dynamic facial and vocal expression features using partial least square regression. In Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge. ACM, 21--30.

Digital Library

[25]

Ali Mollahosseini, David Chan, and Mohammad H Mahoor. 2016. Going deeper in facial expression recognition using deep neural networks. In Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 1--10.

[26]

Qin Ni, Ana Belén García Hernando, and Iván Pau de la Cruz. 2015. The elderly's independent living in smart homes: A characterization of activities and sensing infrastructure survey to facilitate services development. Sensors 15, 5 (2015), 11312--11362.

[27]

Ebenezer Owusu, Yongzhao Zhan, and Qi Rong Mao. 2014. A neural-AdaBoost based facial expression recognition system. Expert Systems with Applications 41, 7 (2014), 3383--3390.

Digital Library

[28]

Ming-Zher Poh, Nicholas C Swenson, and Rosalind W Picard. 2010. A wearable sensor for unobtrusive, long-term assessment of electrodermal activity. IEEE transactions on Biomedical engineering 57, 5 (2010), 1243--1252.

[29]

PD Robert, P Juszczak, P Paclik, et al. 2004. PRTools4, A Matlab Toolbox for Pattern Recognition. Delft University of Technology (2004).

[30]

Mohammad Soleymani, Jeroen Lichtenauer, Thierry Pun, and Maja Pantic. 2012. A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing 3, 1 (2012), 42--55.

Digital Library

[31]

Samarth Tripathi, Shrinivas Acharya, Ranti Dev Sharma, Sudhanshu Mittal, and Samit Bhattacharya. 2017. Using Deep and Convolutional Neural Networks for Accurate Emotion Classification on DEAP Dataset. In AAAI. 4746--4752.

[32]

Md Zia Uddin, Mohammad Mehedi Hassan, Ahmad Almogren, Atif Alamri, Majed Alrubaian, and Giancarlo Fortino. 2017. Facial expression recognition utilizing local direction-based robust features and deep belief network. IEEE Access 5 (2017), 4525--4536.

[33]

Gert Van Der Vloed and Jelle Berentsen. 2009. Measuring emotional wellbeing with a non-intrusive bed sensor. Human-Computer Interaction-INTERACT 2009 (2009), 908--911.

Digital Library

[34]

Gyanendra K Verma and Uma Shanker Tiwary. 2014. Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals. NeuroImage 102 (2014), 162--172.

[35]

Wanhui Wen, Guangyuan Liu, Nanpu Cheng, Jie Wei, Pengchao Shangguan, and Wenjin Huang. 2014. Emotion recognition based on multi-variant correlation of physiological signals. IEEE Transactions on Affective Computing 5, 2 (2014), 126--140.

[36]

Carl E Williams and Kenneth N Stevens. 1981. Vocal correlates of emotional states. Speech evaluation in psychiatry (1981), 221--240.

[37]

Yaqian Xu, Isabel Hübener, Ann-Kathrin Seipp, Sandra Ohly, and Klaus David. 2017. From the lab to the real-world: An investigation on the influence of human movement on Emotion Recognition using physiological signals. In Pervasive Computing and Communications Workshops (PerCom Workshops), 2017 IEEE International Conference on. IEEE, 345--350.

[38]

Zhiding Yu and Cha Zhang. 2015. Image based static facial expression recognition with multiple deep network learning. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, 435--442.

Digital Library

Cited By

Yang KTag BWang CGu YSarsenbayeva ZDingler TWadley GGoncalves J(2023)Survey on Emotion Sensing Using Mobile DevicesIEEE Transactions on Affective Computing10.1109/TAFFC.2022.322048414:4(2678-2696)Online publication date: 1-Oct-2023
https://doi.org/10.1109/TAFFC.2022.3220484
Anupama Save VBiswas AKumar VLourens MMadhura KSharma A(2023)Deep Learning Approach For Emotions DetectionE3S Web of Conferences10.1051/e3sconf/202339907007399(07007)Online publication date: 12-Jul-2023
https://doi.org/10.1051/e3sconf/202339907007
Zaman KSun ZShah SShoaib MPei LHussain A(2022)Driver Emotions Recognition Based on Improved Faster R-CNN and Neural Architectural Search NetworkSymmetry10.3390/sym1404068714:4(687)Online publication date: 26-Mar-2022
https://doi.org/10.3390/sym14040687
Show More Cited By

Recommendations

Human-Computer Interaction Using Emotion Recognition from Facial Expression
EMS '11: Proceedings of the 2011 UKSim 5th European Symposium on Computer Modeling and Simulation

This paper describes emotion recognition system based on facial expression. A fully automatic facial expression recognition system is based on three steps: face detection, facial characteristic extraction and facial expression classification. We have ...
EmotiW 2016: video and group-level emotion recognition challenges
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

This paper discusses the baseline for the Emotion Recognition in the Wild (EmotiW) 2016 challenge. Continuing on the theme of automatic affect recognition `in the wild', the EmotiW challenge 2016 consists of two sub-challenges: an audio-video based ...
Emotion recognition in the wild from videos using images
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

This paper presents the implementation details of the proposed solution to the Emotion Recognition in the Wild 2016 Challenge, in the category of video-based emotion recognition. The proposed approach takes the video stream from the audio-video trimmed ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PETRA '18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

June 2018

591 pages

ISBN:9781450363907

DOI:10.1145/3197768

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

NSF: National Science Foundation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

PETRA '18

PETRA '18: The 11th PErvasive Technologies Related to Assistive Environments Conference

June 26 - 29, 2018

Corfu, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
218
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)3

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang KTag BWang CGu YSarsenbayeva ZDingler TWadley GGoncalves J(2023)Survey on Emotion Sensing Using Mobile DevicesIEEE Transactions on Affective Computing10.1109/TAFFC.2022.322048414:4(2678-2696)Online publication date: 1-Oct-2023
https://doi.org/10.1109/TAFFC.2022.3220484
Anupama Save VBiswas AKumar VLourens MMadhura KSharma A(2023)Deep Learning Approach For Emotions DetectionE3S Web of Conferences10.1051/e3sconf/202339907007399(07007)Online publication date: 12-Jul-2023
https://doi.org/10.1051/e3sconf/202339907007
Zaman KSun ZShah SShoaib MPei LHussain A(2022)Driver Emotions Recognition Based on Improved Faster R-CNN and Neural Architectural Search NetworkSymmetry10.3390/sym1404068714:4(687)Online publication date: 26-Mar-2022
https://doi.org/10.3390/sym14040687
de Santana VOtani LMonteiro Ida Hora Rodrigues Kde Gois Ribeiro Darin TFreire AMota M(2021)Measuring Quantitative Situated User Experience with a Mobile Galvanic Skin Response SensorProceedings of the XX Brazilian Symposium on Human Factors in Computing Systems10.1145/3472301.3484339(1-7)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3472301.3484339
Bulagang AMountstephens JTeo J(2021)A Novel Approach for Emotion Classification in Virtual Reality using Heart Rate (HR) and Inter-beat Interval (IBI)2021 IEEE International Conference on Computing (ICOCO)10.1109/ICOCO53166.2021.9673506(247-252)Online publication date: 17-Nov-2021
https://doi.org/10.1109/ICOCO53166.2021.9673506
Bartolomé-Tomás ASánchez-Reolid RFernández-Sotos ALatorre JFernández-Caballero A(2020)Arousal Detection in Elderly People from Electrodermal Activity Using Musical StimuliSensors10.3390/s2017478820:17(4788)Online publication date: 25-Aug-2020
https://doi.org/10.3390/s20174788
Fiorini LMancioppi GSemeraro FFujita HCavallo F(2020)Unsupervised emotional state classification through physiological parameters for social robotics applicationsKnowledge-Based Systems10.1016/j.knosys.2019.105217190:COnline publication date: 29-Feb-2020
https://dl.acm.org/doi/10.1016/j.knosys.2019.105217
Al Machot FElmachot AAli MAl Machot EKyamakya K(2019)A Deep-Learning Model for Subject-Independent Human Emotion Recognition Using Electrodermal Activity SensorsSensors10.3390/s1907165919:7(1659)Online publication date: 7-Apr-2019
https://doi.org/10.3390/s19071659
Cavallo FSemeraro FMancioppi GBetti SFiorini L(2019)Mood classification through physiological parametersJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-019-01595-6Online publication date: 7-Dec-2019
https://doi.org/10.1007/s12652-019-01595-6

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents