Emotional Debiasing Explanations for Decisions in HCI
Authors: 
Christian Schütze, Olesja Lammert, Birte Richter, Kirsten Thommes, Britta WredeAuthors Info & Claims
Christian Schütze, Olesja Lammert, Birte Richter, Kirsten Thommes, Britta WredeAuthors Info & ClaimsPages 318 - 336
Abstract
Emotions play an important role in human decision-making. However, first approaches to incorporating knowledge of this influence into AI-based decision support systems are only very recent. Accordingly, our target is to develop an interactive intelligent agent that is capable of explaining the recommendations of AI-systems while taking emotional constraints into account. This article addresses the following research questions based on the emotions of happiness and anxiety: (1) How do induced emotions influence risk propensity in HCI? (2) To what extent does the explanation strategy influence the human explanation recipient in a lottery choice? (3) How well can an HCI system estimate the emotional state of the human? Our results showed that (1) our emotion induction strategy was successful. However, the trend took the opposite direction of ATF predictions. (2) Our explanation strategy yielded a change in the risk decision in only 26% of the participants; in some cases, participants even changed their selection in the opposite direction. (3) Emotion recognition from facial expressions did not provide sufficient indications of the emotional state - because of head position and a lack of emotional display - but heart rate showed significant effects of emotion induction in the expected direction. Importantly, in individual cases, the dynamics of facial expressions followed the expected path. We concluded that (1) more differentiated explanation strategies are needed, and that temporal dynamics may play an important role in the explanation process, and (2) that a more interactive setting is required to elicit more emotional cues that can be used to adapt the explanation strategy accordingly.
References
[1]
Angie AD, Connelly S, Waples EP, and Kligyte V The influence of discrete emotions on judgement and decision-making: a meta-analytic review Cogn. Emotion 2011 25 8 1393-1422
[2]
Azarbarzin A, Ostrowski M, Hanly P, and Younes M Relationship between arousal intensity and heart rate response to arousal Sleep 2014 37 4 645-653
[3]
Bradley MM and Lang PJ Measuring emotion: the self-assessment manikin and the semantic differential J. Behav. Ther. Exp. Psychiatry 1994 25 1 49-59
[4]
Chaminade T, Zecca M, Blakemore SJ, Takanishi A, Frith CD, Micera S, Dario P, Rizzolatti G, Gallese V, and Umiltà MA Brain response to a humanoid robot in areas implicated in the perception of human emotional gestures PLoS ONE 2010 5 7
[5]
Critchley HD, Rotshtein P, Nagai Y, O’Doherty J, Mathias CJ, and Dolan RJ Activity in the human brain predicting differential heart rate responses to emotional facial expressions Neuroimage 2005 24 3 751-762
[6]
Crivelli, C., Fridlund, A.: Facial displays are tools for social influence. Trends Cognitive Sci. 22(5) (2018)
[7]
Ekman P Universal and cultural differences in facial expressions of emotions Nebraksa Symposium Motivation 1971 19 207-283
[8]
Elliott, M.V., Johnson, S.L., Pearlstein, J.G., Lopez, D.E.M., Keren, H.: Emotion-related impulsivity and risky decision-making: a systematic review and meta-regression. Clinical Psychol. Rev., 102232 (2022)
[9]
Eyben, F., Wöllmer, M., Schuller, B.: Opensmile: the Munich versatile and fast open-source audio feature extractor. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 1459–1462 (2010)
[10]
Franke T, Attig C, and Wessel D A personal resource for technology interaction: development and validation of the affinity for technology interaction (ati) scale Int. J. Hum.-Comput. Interact. 2019 35 6 456-467
[11]
Fredrickson, B.L.: Positive emotions broaden and build. In: Advances in Experimental Social Psychology, vol. 47, pp. 1–53. Elsevier (2013)
[12]
Hess U, Banse R, and Kappas A The intensity of facial expression is determined by underlying affective state and social situation J. Personlaity Soc. Psychol. 1995 69 2 280-288
[13]
Holt CA and Laury SK Risk aversion and incentive effects Am. Econ. Rev. 2002 92 5 1644-1655
[14]
Jaiswal S, Virmani S, Sethi V, De K, and Roy PP An intelligent recommendation system using gaze and emotion detection Multimed. Tools Appl. 2019 78 14231-14250
[15]
Kensinger EA Remembering the details: effects of emotion Emot. Rev. 2009 2 1 99-113
[16]
Leiner, D. J.: Sosci survey. https://www.soscisurvey.de
[17]
Lerner JS, Han S, and Keltner D Feelings and consumer decision making: extending the appraisal-tendency framework J. Consum. Psychol. 2007 17 3 181-187
[18]
Lerner JS and Keltner D Beyond valence: toward a model of emotion-specific influences on judgement and choice Cogn. Emotion 2000 14 4 473-493
[19]
Lerner JS and Keltner D Fear, anger, and risk J. Pers. Soc. Psychol. 2001 81 1 146
[20]
Lerner JS, Li Y, Valdesolo P, and Kassam KS Emotion and decision making Annu. Rev. Psychol. 2015 66 799-823
[21]
Lerner JS and Tiedens LZ Portrait of the angry decision maker: how appraisal tendencies shape anger’s influence on cognition J. Behav. Decis. Mak. 2006 19 2 115-137
[22]
Lütkebohle, I., et al.: The bielefeld anthropomorphic robot head “flobi”. In: 2010 IEEE International Conference on Robotics and Automation, pp. 3384–3391 (2010).
[23]
Mills C and D’Mello S On the validity of the autobiographical emotional memory task for emotion induction PLoS ONE 2014 9 4
[24]
Moscato V, Picariello A, and Sperlí G An emotional recommender system for music IEEE Intell. Syst. 2021 36 5 57-68
[25]
Rosenthal-von der Pütten, A.M., Krämer, N.C., Hoffmann, L., Sobieraj, S., Eimler, S.C.: An experimental study on emotional reactions towards a robot. Int. J. Soc. Robot. 5(1), 17–34 (2013)
[26]
Rabin M and Thaler RH Anomalies: risk aversion J. Econ. Perspectives 2001 15 1 219-232
[27]
Rammstedt, B., Kemper, C.J., Klein, M.C., Beierlein, C., Kovaleva, A.: Big five inventory (bfi-10). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS) (2014). https://zis.gesis.org/DoiId/zis76
[28]
Ribeiro FS, Santos FH, Albuquerque PB, and Oliveira-Silva P Emotional induction through music: measuring cardiac and electrodermal responses of emotional states and their persistence Front. Psychol. 2019 10 451
[29]
Riedl MO Human-centered artificial intelligence and machine learning Hum. Behav. Emerg. Technol. 2019 1 1 33-36
[30]
Rohlfing KJ et al. Explanation as a social practice: toward a conceptual framework for the social design of AI systems IEEE Trans. Cogn. Dev. Syst. 2020 13 3 717-728
[31]
Ruiz-Belda MA, Fernández-Dols JM, Carrera P, and Barchard K Spontaneous facial expressions of happy bowlers and soccer fans Cogn. Emot. 2003 17 2 315-326
[32]
Schulreich S, Gerhardt H, and Heekeren HR Incidental fear cues increase monetary loss aversion Emotion 2016 16 3 402
[33]
Serengil, S.I., Ozpinar, A.: Hyperextended lightface: a facial attribute analysis framework. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–4. IEEE (2021).
[34]
Siedlecka E and Denson TF Experimental methods for inducing basic emotions: a qualitative review Emot. Rev. 2019 11 1 87-97
[35]
Smith CA and Ellsworth PC Patterns of cognitive appraisal in emotion J. Pers. Soc. Psychol. 1985 48 4 813
[36]
Spielberger, C.D.: Manual for the state-trait anxietry, inventory. Consulting Psychologist (1970)
[37]
Thiruchselvam, R., Blechert, J., Sheppes, G., Rydstrom, A., Gross, J.J.: The temporal dynamics of emotion regulation: an eeg study of distraction and reappraisal. Biol. Psychol. 87(1), 84–92 (2011). https://www.sciencedirect.com/science/article/pii/S0301051111000391
[38]
Toisoul, A., Kossaifi, J., Bulat, A., Tzimiropoulos, G., Pantic, M.: Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nature Mach. Intell. (2021). https://www.nature.com/articles/s42256-020-00280-0
[39]
Wang, D., Yang, Q., Abdul, A., Lim, B.Y.: Designing theory-driven user-centric explainable AI. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2019)
[40]
Yang Q, Zhou S, Gu R, and Wu Y How do different kinds of incidental emotions influence risk decision making? Biol. Psychol. 2020 154
Recommendations
Emojis influence emotional communication, social attributions, and information processing
AbstractMany emojis symbolize nonverbal cues that are used during face-to-face communication. Despite their popularity, few studies have examined how emojis influence digital interactions. The present study addresses this gap by measuring the ...
Highlights- Emojis convey information about the sender's affect.
- Senders that use positive ...
Emotional clouds: Showing arousal and valence through the movement and darkness of digital cartoonish clouds
While nowadays the most usual way to show emotions in digital contexts is via virtual characters, its use may raise false expectations (the user attributes human abilities to the virtual character). This paper proposes and explores an approach to ...
Emotional responses to watching and touching 3d emotional face in a virtual environment
AbstractFacial expressions play a crucial role in modulating the emotional responses in the viewers. Touch is an important factor in shaping human emotions and social communication. The objective of this study is to investigate the effects of viewing and ...
Comments
Information & Contributors
Information
Published In
Jul 2023
682 pages
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 23 July 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 31 Jan 2025