research-article

Implicit User-centric Personality Recognition Based on Physiological Responses to Emotional Videos

Authors:

Ramanathan Subramanian,

Mojtaba Khomami Abadi,

Radu-Laurentiu Vieriu,

Stefan WinklerAuthors Info & Claims

ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

Pages 239 - 246

https://doi.org/10.1145/2818346.2820736

Published: 09 November 2015 Publication History

Abstract

We present a novel framework for recognizing personality traits based on users' physiological responses to affective movie clips. Extending studies that have correlated explicit/implicit affective user responses with Extraversion and Neuroticism traits, we perform single-trial recognition of the big-five traits from Electrocardiogram (ECG), Galvanic Skin Response (GSR), Electroencephalogram (EEG) and facial emotional responses compiled from 36 users using off-the-shelf sensors. Firstly, we examine relationships among personality scales and (explicit) affective user ratings acquired in the context of prior observations. Secondly, we isolate physiological correlates of personality traits. Finally, unimodal and multimodal personality recognition results are presented. Personality differences are better revealed while analyzing responses to emotionally homogeneous (e.g., high valence, high arousal) clips, and significantly above-chance recognition is achieved for all five traits.

References

[1]

M. K. Abadi, J. A. M. Correa, J. Wache, H. Yang, I. Patras, and N. Sebe. Inference of Personality Traits and Affect Schedule by Analysis of Spontaneous Reactions to Affective Videos. In FG, 2015.

[2]

M. K. Abadi, S. M. Kia, R. Subramanian, P. Avesani, and N. Sebe. User-centric Affective Video Tagging from MEG and Peripheral Physiological Responses. In ACII, 2013.

Digital Library

[3]

M. K. Abadi, R. Subramanian, S. Kia, P. Avesani, I. Patras, and N. Sebe. DECAF: MEG-based Multimodal Database for Decoding Affective Physiological Responses. IEEE Trans. Affective Computing, 2015.

[4]

S. Argamon, S. Dhawle, M. Koppel, and Pennbaker. Lexical predictors of personality type. In Interface and the Classification Society of North America, 2005.

[5]

L. M. Batrinca, N. Mana, B. Lepri, F. Pianesi, and N. Sebe. Please, tell me about yourself: Automatic Personality Assessment Using Short Self-Presentations. In ICMI, 2011.

Digital Library

[6]

L. Brown, B. Grundlehner, and J. Penders. Towards wireless emotional valence detection from EEG. In EMBS, pages 2188--2191, 2011.

[7]

P. T. Costa and R. R. McCrae. Influence of Extraversion and Neuroticism on Subjective Well-Being: Happy and Unhappy People. Journal of Personality and Social Psychology, 38(4):668, 1980.

[8]

P. T. J. Costa and R. R. McCrae. NEO-PI-R Professional Manual: Revised NEO Personality and NEO Five-Factor Inventory (NEO-FFI), volume 4. Psychological Assessment Resources, 1992.

[9]

V. De Pascalis, E. Strippoli, P. Riccardi, and F. Vergari. Personality, Event-Related Potential (ERP) and Heart Rate (HR) in Emotional Word Processing. Pers. Individ. Dif., 36:873--891, 2004.

[10]

H. J. Eysenck. Dimensions of Personality, volume 5. Transaction Publishers, 1950.

[11]

S. Gupta and J. Nicholson. Simple visual reaction time, personality strength of the nervous system: theory approach. Pers. Individ. Dif., 6(4):461--469, 1985.

[12]

H. Joho, J. Staiano, N. Sebe, and J. M. Jose. Looking at the viewer: analysing facial activity to detect personal highlights of multimedia contents. Multimedia Tools Appl., 51(2):505--523, 2011.

Digital Library

[13]

E. G. Kehoe, J. M. Toomey, J. H. Balsters, and A. L. W. Bokde. Personality modulates the effects of emotional arousal and valence on brain activation. Soc. Cogn. Affect. Neurosci., 7:858--70, 2012.

[14]

J. Kim and E. Andre. Emotion recognition based on physiological changes in music listening. IEEE TPAMI, 30(12):2067--2083, 2008.

Digital Library

[15]

S. Koelstra, C. Mühl, M. Soleymani, J.-S. Lee, A. Yazdan, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras. DEAP: A Database for Emotion Analysis Using Physiological Signals. IEEE Trans. Affective Computing, 3(1):18--31, 2012.

Digital Library

[16]

S. Koelstra and I. Patras. Fusion of facial expressions and EEG for implicit affective tagging. Image and Vision Computing, 31(2):164--174, 2013.

Digital Library

[17]

B. Lepri, R. Subramanian, K. Kalimeri, J. Staiano, F. Pianesi, and N. Sebe. Connecting meeting behavior with extraversion - a systematic study. IEEE Trans. Affective Computing, 3(4):443--455, 2012.

Digital Library

[18]

C. L. Lisetti and F. Nasoz. Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP J. Adv. Sig. Proc., 2004(11):1672--1687, 2004.

Digital Library

[19]

F. Mairesse, M. A. Walker, M. R. Mehl, and R. K. Moore. Using linguistic cues for the automatic recognition of personality in conversation and text. Journal of Artificial Intelligence Research, 30:457--500, 2007.

Digital Library

[20]

R. R. McCrae and P. T. Costa. A contemplated revision of the neo five-factor inventory. Pers. Individ. Dif., 36(3):587--596, 2004.

[21]

I. Mervielde, F. De Fruyt, and S. Jarmuz. Linking openness and intellect in childhood and adulthood. Parental descriptions of child personality: Developmental antecedents of the Big Five, pages 105--126, 1998.

[22]

W. Ng. Clarifying the relation between neuroticism and positive emotions. Pers. Individ. Dif., 47(1):69--72, 2009.

[23]

D. Olguin, B. Waber, T. Kim, A. Mohan, K. Ara, and A. Pentland. Sensible organizations: Technology and methodology for automatically measuring organizational behavior. IEEE Trans. Systems, Man, and Cybernetics, 39(1):43--55, 2009.

Digital Library

[24]

M. Perugini and L. Di Blas. Analyzing Personality-Related Adjectives from an Eticemic Perspective: the Big Five Marker Scale (BFMS) and the Italian AB5C Taxonomy. Big Five Assessment, pages 281--304, 2002.

[25]

M. Soleymani, J. Lichtenauer, T. Pun, and M. Pantic. A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affective Computing, 3:42--55, 2012.

Digital Library

[26]

R. Srivastava, J. Feng, S. Roy, S. Yan, and T. Sim. Don't Ask Me What I'm Like, Just Watch and Listen. In ACMMM, 2012.

Digital Library

[27]

G. Stenberg. Personality and the EEG: Arousal and emotional arousability. Pers. Individ. Dif., 13:1097--1113, 1992.

[28]

R. Subramanian, Y. Yan, J. Staiano, O. Lanz, and N. Sebe. On the relationship between head pose, social attention and personality prediction for unstructured and dynamic group interactions. In ICMI, 2013.

Digital Library

[29]

S. Tok, M. Koyuncu, S. Dural, and F. Catikkas. Evaluation of International Affective Picture System (IAPS) ratings in an athlete population and its relations to personality. Pers. Individ. Dif., 49(5):461--466, 2010.

[30]

A. Vinciarelli and G. Mohammadi. A survey of personality computing. IEEE Trans. Affective Computing, (3):273--291, 2014.

[31]

G. Zen, B. Lepri, E. Ricci, and O. Lanz. Space speaks: towards socially and personality aware visual surveillance. In ACMMPVA, pages 37--42, 2010.

Digital Library

[32]

Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE TPAMI, 31(1):39--58, 2009.

Digital Library

Cited By

Banik SSen SSaha SGhosh S(2024)Towards Reducing Continuous Emotion Annotation Effort During Video Consumption: A Physiological Response Profiling ApproachProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785698:3(1-32)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678569
Riaz MGravina R(2024)Wearable Sensor Systems to Detect Biomarkers of Personality Traits for Healthy Aging: A ReviewIEEE Sensors Journal10.1109/JSEN.2024.342935724:17(27061-27075)Online publication date: 1-Sep-2024
https://doi.org/10.1109/JSEN.2024.3429357
Mathur LMataric MMorency L(2023)Expanding the Role of Affective Phenomena in Multimodal Interaction ResearchProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614171(253-260)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3614171
Show More Cited By

Index Terms

Implicit User-centric Personality Recognition Based on Physiological Responses to Emotional Videos
1. Computing methodologies
  1. Machine learning
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Towards a Model for Personality-Based Agents for Emotional Responses
Webmedia '16: Proceedings of the 22nd Brazilian Symposium on Multimedia and the Web

Affective Computing is a promising research area with many open challenges. This area expects to develop computational systems that can monitor and respond to the affective states of an interacting user (IU). These affective states can be observed in ...
Comparative analysis of affective and physiological responses to emotional movies

In this research, we investigated on user's affective and physiological responses to the emotional movies. The emotional movies provide continuous feedback such as changes in saturation, brightness and contrast of the movie in response to users' ...
Multimodal recognition of personality traits in human-computer collaborative tasks
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interaction

The user's personality in Human-Computer Interaction (HCI) plays an important role for the overall success of the interaction. The present study focuses on automatically recognizing the Big Five personality traits from 2-5 min long videos, in which the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

November 2015

678 pages

ISBN:9781450339124

DOI:10.1145/2818346

General Chairs:
Zhengyou Zhang
Microsoft Research, USA
,
Phil Cohen
VoiceBox Technologies, USA
,
Program Chairs:
Dan Bohus
Microsoft Research, USA
,
Radu Horaud
INRIA Grenoble Rhone-Alpes, France
,
Helen Meng
Chinese University of Hong Kong, China

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Singapore's Agency for Science Technology and Research
FP7 European Project

Conference

ICMI '15

Sponsor:

SIGCHI

ICMI '15: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 9 - 13, 2015

Washington, Seattle, USA

Acceptance Rates

ICMI '15 Paper Acceptance Rate 52 of 127 submissions, 41%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
404
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)3

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Banik SSen SSaha SGhosh S(2024)Towards Reducing Continuous Emotion Annotation Effort During Video Consumption: A Physiological Response Profiling ApproachProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785698:3(1-32)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678569
Riaz MGravina R(2024)Wearable Sensor Systems to Detect Biomarkers of Personality Traits for Healthy Aging: A ReviewIEEE Sensors Journal10.1109/JSEN.2024.342935724:17(27061-27075)Online publication date: 1-Sep-2024
https://doi.org/10.1109/JSEN.2024.3429357
Mathur LMataric MMorency L(2023)Expanding the Role of Affective Phenomena in Multimodal Interaction ResearchProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614171(253-260)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3614171
Yang HLee C(2023)A Media-Guided Attentive Graphical Network for Personality Recognition Using PhysiologyIEEE Transactions on Affective Computing10.1109/TAFFC.2021.309004014:2(931-943)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TAFFC.2021.3090040
Biswas ALi JShubho FGedeon TRahman SHossain M(2023)Personality Perception Using Scenario Based Stimulation and Physiological Signals2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394366(1884-1891)Online publication date: 1-Oct-2023
https://doi.org/10.1109/SMC53992.2023.10394366
Pan YXu J(2023)Personality Traits Inference in the Hybrid Foraging Search TaskDesign, User Experience, and Usability10.1007/978-3-031-35702-2_19(258-269)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35702-2_19
Alves TRamalho BGonçalves DHenriques-Calado JGama S(2023)Leveraging Conscientiousness-Based Preferences in Information Visualization DesignVisualization Psychology10.1007/978-3-031-34738-2_13(293-330)Online publication date: 7-Nov-2023
https://doi.org/10.1007/978-3-031-34738-2_13
Khan ARasool S(2022)Game induced emotion analysis using electroencephalographyComputers in Biology and Medicine10.1016/j.compbiomed.2022.105441145:COnline publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2022.105441
Chen LCai WYan DBerkovsky S(2022)Eye-tracking-based personality prediction with recommendation interfacesUser Modeling and User-Adapted Interaction10.1007/s11257-022-09336-933:1(121-157)Online publication date: 24-Jun-2022
https://doi.org/10.1007/s11257-022-09336-9
Alves TNunes DGonçalves DHenriques-Calado JGama S(2022)Towards conscientiousness-based graphical user interface design guidelinesPersonal and Ubiquitous Computing10.1007/s00779-022-01693-x27:2(401-414)Online publication date: 10-Nov-2022
https://doi.org/10.1007/s00779-022-01693-x
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten