research-article

Emotional facial expression interface: effects of displayed facial designs

Authors:

Nadine Mandran,

Anna TcherkassofAuthors Info & Claims

Ergo'IA '10: Proceedings of the Ergonomie et Informatique Avancee Conference

Pages 37 - 43

https://doi.org/10.1145/1868650.1868657

Published: 12 October 2010 Publication History

Abstract

In supporting the visual-mediated emotional recognition, little research has centred on analysing the effect of presenting different combinatorial facial designs. Moreover, in the theoretical field of emotions, research on the recognition of emotional facial expressions (EFE) is mainly based on static and posed databases tested in unnatural contexts (explicit recognition task of an emotion).

Our research has constructed and validated a database, DynEmo, with dynamic and spontaneous emotional facial expressions. So different facial interface designs (whole face, zoomed face, eyes + mouth, whole face + mouth, whole face + eyes, etc., n = 11) are experimentally compared to find their impact in terms of emotional recognition.

The results show the facial interface design relevance in terms of emotional recognition. The application of this work is transferable to facial interfaces useful for video-mediated interaction.

References

[1]

Ambadar, Z., Schooler, J., and Cohn, J. 2005. Deciphering the enigmatic face: The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16, 403--410.

[2]

Anderson A. H., Smallwood L., MacDonald R., Mullin J., Fleming A. M., and O'Malley C. 1999. Video data and video links in mediated communication: What do users value? International journal of human-computer studies, 2, 165--187.

Digital Library

[3]

Back, E., Jordan, T. R., and Thomas, S. M. 2009. The recognition of mental states from dynamic and static facial expressions. Visual cognition, 17, 1271--1286.

[4]

Bassili, J. N. 1978. Facial motion in the perception of faces and of emotional expression. Journal of Experimental Psychology: Human Perception and Performance, 4, 373--379.

[5]

Bassili, J. N. 1979. Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. Journal of Personality and Social Psychology, 37, 2049--2058.

[6]

Bétrancourt, M., Dillenbourg, P., and Clavien, L. 2008. Reducing cognitive load in learning from animations: Impacts of delivery features. In J.-F. Rouet, W. Schnotz and R. Lowe Eds., Understanding Multimedia Documents pp 61--78.

[7]

Dupré, D. and Tcherkassof, A. 2010. On-line Recognition of Dynamic and Spontaneous Facial Expression. Proceedings of the Consortium of European Research on Emotion, Villeneuve d'Ascq, France, April 23.

[8]

Ekman, P. and Friesen, W. V. 1976. Pictures of facial affect. Palo Alto, C. A.: Consulting Psychologists Press.

[9]

Fernández-Dols, J.-M., Sánchez, F., Carrera, P., and Ruiz-Belda, M.-A. 1997. Are spontaneous expressions and emotions linked? An experimental test of coherence. Journal of Nonverbal Behavior, 21, 163--177.

[10]

Heath C. and Luff, P. 1991. Disembodied conduct: Communication through video in multi-media office environment. Proceedings of CHI'91 pp. 99--103, New Orleans: ACM press.

Digital Library

[11]

Hess, U. and Kleck, R. E. 1990. Differentiating emotion elicited and deliberate emotional facial expressions. European Journal of Social Psychology, 20, 369--385.

[12]

Jamet, E., Bétrancourt, M., and Rouet, J.-F. 2008. La compréhension des documents complexes. In A. Chevalier and A. Tricot Dirs. Ergonomie cognitive des documents électroniques. Paris: Dunod.

[13]

Kamachi, M., Bruce, V., Mukaida, S., Gyoba, J., Yoshikawa, S., and Akamatsu, S. 2001. Dynamic properties influence the perception of facial expressions. Perception, 30, 875--887.

[14]

Kanade, T., Cohn, J. F., and Tian, Y. 2000. Comprehensive database for facial expression analysis. Proceedings of the 4^th IEEE International Conference on Automatic Face and Gesture Recognition FG'00 pp. 46--53, March 2000, Grenoble, France.

Digital Library

[15]

Lemay, G., Kirouac, G., and Lacouture, Y. 1995. Expressions faciales émotionnelles spontanées, dynamiques et statiques: comparaison d'études de jugement catégoriel et dimensionnel. {Dynamic and static spontaneous emotional facial expressions: A comparison of categorical and dimensional judgment studies}. Canadian Journal of Behavioral Science, 27, 125--139.

[16]

Manstead, A. S. R., Fisher, A. H., and Jakobs, E. B. 1999. The social and emotional functions of facial displays. In P. Philippot, R. S. Feldman and E. J. Coats Eds. The social context of nonverbal behavior pp. 287--313. Cambridge, UK: Cambridge University Press.

[17]

Matsumoto, D. and Ekman, P. 1988. Japanese and Caucasian Facial Expressions of Emotion and Neutral Faces JACFEE and JACNeuF. Available from Human Interaction Laboratory, University of California, San Francisco, 401 Parnassus Avenue, San Francisco, CA, 94143.

[18]

Mayer, R. E. and Anderson, R. B. 1992. The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84, 444--452.

[19]

Meillon, B., Tcherkassof, A., Mandran, N., Adam, J.-M., Dubois, M., Dupré, D., Benoît, A.-M., Guérin-Dugué, A., Caplier, A. 2010. DYNEMO: A Corpus of dynamic and spontaneous emotional facial expressions. Communication orale à paraître dans Proceedings of the International Workshop Series on Multimodal Corpora, Tools and Resources: Advances in Capturing, Coding and Analyzing Multimodality LREC, Malte, 18 Mai.

[20]

Motley, M. T. and Camden, C. T. 1988. Facial expression of emotion: A comparison of posed versus spontaneous expressions in an interpersonal communication setting. Western Journal of Speech Communication, 52, 1--22.

[21]

Nummenmaa, T. 1992. Pure and blended emotion in the face. Helsinki, Finland: Academia Scientiarum Fennica.

[22]

O'Conaill B., Whittaker S., and Wilbur S., 1994. Making the case for face-to-face interaction: Empirical studies of video-mediated interaction. Human Computer Interaction, 8, 389--428.

Digital Library

[23]

O'Malley C., Langton, S., Anderson, A., Doherty-Sneddon, G., and Bruce, V. 1996. Comparison of face-to-face and video-mediated interaction. Interacting with computers, 8, 177--192.

Digital Library

[24]

Russell, J. A. 1994. Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115, 102--141.

[25]

Russell, J. A. 1997. Reading emotions from and into faces: Resurrecting a dimensional-contextual perspective. In J. A. Russell and J. M. Fernández-Dols Eds., The psychology of facial expression pp. 295--320. Cambridge, UK: Cambridge University Press.

[26]

Rutter, D. and Stephenson G., 1977. The role of visual communication in synchronising conversation. European Journal of Social Psychology, 7, 29--37

[27]

Short, J., Williams, E., and Christi, B. 1976. The social psychology of telecommunications. New York: Wiley.

[28]

Sweller, J. and Chandler, P. 1994. Why some material is difficult to learn. Cognition and Instruction, 12, 185--233.

[29]

Tajariol F., Adam, Michel, J. M., and Dubois, M. 2008. Seeing the face and observing the actions: The effects of nonverbal cues on mediated tutoring dialogue. In E. Aimeur, and B. Woolf Eds. Proceedings of the 9th International Conference on Intelligent Tutoring Systems. Berlin, Germany.

Digital Library

[30]

Tajariol, F., Adam, J. M., and Dubois, M. 2009. The impact of nonverbal cues on mediated tutoring interaction: an experimental study. In press.

[31]

Tcherkassof A., Bollon T., Dubois M., Pansu P., and Adam J.-M. 2007. Facial expressions of emotions: A methodological contribution to the study of spontaneous and dynamic emotional faces. European Journal of Social Psychology, 37, 1325--1345.

[32]

Wallbott, H. G. and Scherer, K. R. 1986. Cues and channels in emotion recognition. Journal of Personality and Social Psychology, 51, 690--699.

Cited By

Freire CVergara-Perucich FJullian Suárez CBerridge G(2022)Utilización de patrones respiratorios en la transmisión de emociones mixtas en personajes animadosCon A de animación10.4995/caa.2022.17144(126-141)Online publication date: 14-Mar-2022
https://doi.org/10.4995/caa.2022.17144
Sarmiento AAviñante RLiwanag JDadiz B(2018)Dimension Reduction Techniques Analysis on Image Processing for Facial Emotion RecognitionProceedings of the 1st International Conference on Control and Computer Vision10.1145/3232651.3232669(23-27)Online publication date: 15-Jun-2018
https://dl.acm.org/doi/10.1145/3232651.3232669
Dubois MDupré DAdam JTcherkassof AMandran NMeillon B(2012)The influence of facial interface design on dynamic emotional recognitionJournal on Multimodal User Interfaces10.1007/s12193-012-0103-y7:1-2(111-119)Online publication date: 1-Aug-2012
https://doi.org/10.1007/s12193-012-0103-y

Index Terms

Emotional facial expression interface: effects of displayed facial designs
1. Computing methodologies
  1. Artificial intelligence
    1. Philosophical/theoretical foundations of artificial intelligence
      1. Cognitive science
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

How to Make a Robot Smile? Perception of Emotional Expressions from Digitally-Extracted Facial Landmark Configurations
Social Robotics
Abstract
To design robots or embodied conversational agents that can accurately display facial expressions indicating an emotional state, we need technology to produce those facial expressions, and research that investigates the relationship between those ...
A bi-modal face recognition framework integrating facial expression with facial appearance

Among many biometric characteristics, the facial biometric is considered to be the least intrusive technology that can be deployed in the real-world visual surveillance environment. However, in facial biometric, little research attention has been paid ...
Natural facial expression recognition using differential-AAM and manifold learning

This paper proposes a novel natural facial expression recognition method that recognizes a sequence of dynamic facial expression images using the differential active appearance model (AAM) and manifold learning as follows. First, the differential-AAM ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

Ergo'IA '10: Proceedings of the Ergonomie et Informatique Avancee Conference

October 2010

254 pages

ISBN:9781450302739

DOI:10.1145/1868650

Conference Chairs:
Gaëlle Calvary
Grenoble INP & Laboratoire d'Informatique de Grenoble
,
Marion Wolff
Université; Paris Descartes

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ERGO-IA '10

ERGO-IA '10: Ergonomie et Informatique Avancee: "Innovation, Interactions et Qualité de Vie"

October 12 - 15, 2010

Biarritz, France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
421
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Freire CVergara-Perucich FJullian Suárez CBerridge G(2022)Utilización de patrones respiratorios en la transmisión de emociones mixtas en personajes animadosCon A de animación10.4995/caa.2022.17144(126-141)Online publication date: 14-Mar-2022
https://doi.org/10.4995/caa.2022.17144
Sarmiento AAviñante RLiwanag JDadiz B(2018)Dimension Reduction Techniques Analysis on Image Processing for Facial Emotion RecognitionProceedings of the 1st International Conference on Control and Computer Vision10.1145/3232651.3232669(23-27)Online publication date: 15-Jun-2018
https://dl.acm.org/doi/10.1145/3232651.3232669
Dubois MDupré DAdam JTcherkassof AMandran NMeillon B(2012)The influence of facial interface design on dynamic emotional recognitionJournal on Multimodal User Interfaces10.1007/s12193-012-0103-y7:1-2(111-119)Online publication date: 1-Aug-2012
https://doi.org/10.1007/s12193-012-0103-y

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents