research-article

Inferring Player Experiences Using Facial Expressions Analysis

Authors:

Yusuf PisanAuthors Info & Claims

IE2014: Proceedings of the 2014 Conference on Interactive Entertainment

Pages 1 - 8

https://doi.org/10.1145/2677758.2677765

Published: 02 December 2014 Publication History

Abstract

Understanding player experiences is central to game design. Video captures of players is a common practice for obtaining rich reviewable data for analysing these experiences. However, not enough has been done in investigating ways of preprocessing the video for a more efficient analysis process. This paper consolidates and extends prior work on validating the feasibility of using automated facial expressions analysis as a natural quantitative method for evaluating player experiences. A study was performed on participants playing a first-person puzzle shooter game (Portal 2) and a social drawing trivia game (Draw My Thing), and results were shown to exhibit rich details for inferring player experiences from facial expressions. Significant correlations were also observed between facial expression intensities and self reports from the Game Experience Questionnaire. In particular, the challenge dimension consistently showed positive correlations with anger and joy. This paper eventually presents a case for increasing the application of computer vision in video analyses of gameplay.

References

[1]

Baltrusaitis, T., McDuff, D., Banda, N., Mahmoud, M., Kaliouby, R. E., Robinson, P., and Picard, R. Realtime inference of mental states from facial expressions and upper body gestures. Face and Gesture 2011 (Mar. 2011), 909--914.

[2]

Branco, P. Computer-based facial expression analysis for assessing user experience. PhD thesis, 2006.

[3]

Buenaposada, J. M., Muñoz, E., and Baumela, L. Recognising facial expressions in video sequences. Pattern Analysis and Applications 11, 1 (Oct. 2007), 101--116.

Digital Library

[4]

Cohn, J. F., and Sayette, M. A. Automated Facial Coding. Behavioral Research Methods (2010), 1--33.

[5]

Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience, vol. 54 of P. S. Series. Harper & Row, 1990.

[6]

Drachen, A., Nacke, L., Yannakakis, G., and Pedersen, A. Correlation between heart rate, electrodermal activity and player experience in First-Person Shooter games. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games, ACM (2010), 49--54.

Digital Library

[7]

Ekman, P., and Friesen, W. V. The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1, 1 (1969), 49--98.

[8]

Ekman, P., Friesen, W. V., and Hager, J. C. Facial Action Coding System, vol. 48. A Human Face, 2002.

[9]

Gagne, A., Seif El-Nasr, M., and Shaw, C. A Deeper Look at the use of Telemetry for Analysis of Player Behavior in RTS Games. In International Conference on Entertainment Computing ICEC (2011).

Digital Library

[10]

Ghijsen, M. Facial Expression Analysis for Human Computer Interaction. Affective Computing, IEEE Transactions on 2, 3 (2004), 147--161.

[11]

Isbister, K., and Schaffer, N. Game Usability: Advancing the Player Experience, 1 ed. CRC Press, 2008.

[12]

Mandryk, R. L., Atkins, M. S., and Inkpen, K. M. A continuous and objective evaluation of emotional experience with interactive play environments. Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI '06 (2006), 1027.

Digital Library

[13]

McDuff, D., el Kaliouby, R., Kassam, K., and Picard, R. Acume: A new visualization tool for understanding facial expression and gesture data. In Face and Gesture 2011, IEEE (Mar. 2011), 591--596.

[14]

Medler, B., John, M., and Lane, J. Data cracker: developing a visual game analytic tool for analyzing online gameplay. In Proceedings of the 2011 annual conference on Human factors in computing systems, ACM (2011), 2365--2374.

Digital Library

[15]

Moura, D., Seif El-Nasr, M., and Shaw, C. D. Visualizing and Understanding Players âĂ&Zacute; Behavior in Video Games: Discovering Patterns and Supporting Aggregation and Comparison. In ACM SIGGRAPH Symposium on Video Games (2011), 2--7.

Digital Library

[16]

Nacke, L. E. Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment. PhD thesis, 2009.

[17]

Nacke, L. E., Grimshaw, M. N., and Lindley, C. a. More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game. Interacting with Computers 22, 5 (Sept. 2010), 336--343.

Digital Library

[18]

Nacke, L. E., and Lindley, C. Flow and immersion in first-person shooters: measuring the player's gameplay experience. Proceedings of the 2008 Conference on Future (2008), 81--88.

Digital Library

[19]

Örtqvist, D., and Liljedahl, M. Immersion and Game-play Experience: A Contingency Framework. International Journal of Computer Games Technology 2010 (2010), 1--11.

Digital Library

[20]

Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R. L., and Fuller, T. User-centered design in games. In Design, vol. 28. 2003, 883--906.

Digital Library

[21]

Paris Mavromoustakos Blom, Bakkes, S., Tan, C. T., Whiteson, S., Roijers, D., Valenti, R., and Gevers, T. Towards Personalised Gaming via Facial Expression Recognition. In Proceedings of Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AAAI Press (Palo, 2014).

[22]

Russell, J. A. A circumplex model of affect. Journal of Personality and Social Psychology 39, 6 (1980), 1161--1178.

[23]

Saragih, J. M., Lucey, S., and Cohn, J. F. Deformable Model Fitting by Regularized Landmark Mean-Shift. International Journal of Computer Vision 91, 2 (Sept. 2010), 200--215.

Digital Library

[24]

Tan, C., and Johnston, A. Towards a Nondisruptive, Practical, and Objective Automated Playtesting Process. In Workshops at the Seventh Artificial Intelligence and Interactive Digital Entertainment Conference (2011), 25--28.

[25]

Tan, C. T., Rosser, D., Bakkes, S., and Pisan, Y. A feasibility study in using facial expressions analysis to evaluate player experiences. In Proceedings of The 8th Australasian Conference on Interactive Entertainment, ACM Press (2012), 5.

Digital Library

[26]

Zammitto, V., Seif El-Nasr, M., and Newton, P. Exploring Quantitative Methods for Evaluating Sports Games. Signals (2010), 8--11.

Cited By

Li XWang QWang ZJin ZJia J(2024)SoulSkipper: A Voice-Controlled Emotional Adaptive Game to Complement Therapy for Social Anxiety DisorderExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650822(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650822
Zhang HYin LZhang H(2024)Using subjective emotion, facial expression, and gaze direction to evaluate user affective experience and predict preference when playing single-player gamesErgonomics10.1080/00140139.2024.235912367:12(1863-1883)Online publication date: 4-Jun-2024
https://doi.org/10.1080/00140139.2024.2359123
Zhang HYin LZhang H(2024)A real-time camera-based gaze-tracking system involving dual interactive modes and its application in gamingMultimedia Systems10.1007/s00530-023-01204-930:1Online publication date: 16-Jan-2024
https://dl.acm.org/doi/10.1007/s00530-023-01204-9
Show More Cited By

Index Terms

Inferring Player Experiences Using Facial Expressions Analysis
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information systems applications
    1. Decision support systems
      1. Expert systems

Recommendations

The Convergence of Player Experience Questionnaires
CHI PLAY '16: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play

Player experience is an important field of digital games research to understand how games influence players. A common way to directly measure players' reported experiences is through questionnaires. However, the large number of questionnaires currently ...
Correlating Facial Expressions and Subjective Player Experiences in Competitive Hearthstone
FDG '21: Proceedings of the 16th International Conference on the Foundations of Digital Games

In this study, we used recordings of players’ facial expressions that are captured during competitive Hearthstone games to analyse the correlation between in-game player affective responses and subjective post-game self-reports. With this, we aimed to ...
The interplay between immersion and appeal in video games

A 2í 2 study on immersion and appeal of video games is presented.Immersion and appeal are found to be highly related.Immersion is found to be unaffected by playing experience.Appeal is found to be affected both by game played and playing experience. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IE2014: Proceedings of the 2014 Conference on Interactive Entertainment

December 2014

259 pages

ISBN:9781450327909

DOI:10.1145/2677758

Editors:
Karen Blackmore
University of Newcastle
,
Keith Nesbitt
University of Newcastle
,
Shamus P. Smith
University of Newcastle

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

The University of Newcastle, Australia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 December 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IE2014

IE2014: Interactive Entertainment 2014

December 2 - 3, 2014

NSW, Newcastle, Australia

Acceptance Rates

IE2014 Paper Acceptance Rate 27 of 42 submissions, 64%;

Overall Acceptance Rate 64 of 148 submissions, 43%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
194
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)5

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li XWang QWang ZJin ZJia J(2024)SoulSkipper: A Voice-Controlled Emotional Adaptive Game to Complement Therapy for Social Anxiety DisorderExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650822(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650822
Zhang HYin LZhang H(2024)Using subjective emotion, facial expression, and gaze direction to evaluate user affective experience and predict preference when playing single-player gamesErgonomics10.1080/00140139.2024.235912367:12(1863-1883)Online publication date: 4-Jun-2024
https://doi.org/10.1080/00140139.2024.2359123
Zhang HYin LZhang H(2024)A real-time camera-based gaze-tracking system involving dual interactive modes and its application in gamingMultimedia Systems10.1007/s00530-023-01204-930:1Online publication date: 16-Jan-2024
https://dl.acm.org/doi/10.1007/s00530-023-01204-9
Bandeira ICastanho Ce Silva TSarmet MJacobi R(2023)Applicability of Psychophysiological and Perception Data for Mapping Strategies in League of Legends – An Exploratory StudyHCI in Games10.1007/978-3-031-35979-8_10(125-140)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-35979-8_10
Siqueira EFleury MLamar MDrachen ACastanho CJacobi R(2022)An automated approach to estimate player experience in game events from psychophysiological dataMultimedia Tools and Applications10.1007/s11042-022-13845-582:13(19189-19220)Online publication date: 15-Oct-2022
https://doi.org/10.1007/s11042-022-13845-5
Fleury Me Silva TSarmet MCastanho C(2020)A Tool to Support Players Affective States Assessment Based on Facial Expressions AnalysisHCI in Games10.1007/978-3-030-50164-8_3(39-57)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-50164-8_3
Bevilacqua FEngström HBacklund P(2019)Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in GamesSensors10.3390/s1913287719:13(2877)Online publication date: 28-Jun-2019
https://doi.org/10.3390/s19132877
Ustrup EMathiesen MPoulsen JChristensen JLöchtefeld M(2019)Comparing the Applicability of Pressure as a Game Metric to Self-assessment and Physiological MetricsHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29384-0_24(396-405)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29384-0_24
Bevilacqua FEngström HBacklund P(2018)Automated Analysis of Facial Cues from Videos as a Potential Method for Differentiating Stress and Boredom of Players in GamesInternational Journal of Computer Games Technology10.1155/2018/87345402018(1)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1155/2018/8734540
Siqueira ESantos TCastanho CJacobi R(2018)Estimating Player Experience from Arousal and Valence Using Psychophysiological Signals2018 17th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)10.1109/SBGAMES.2018.00022(107-10709)Online publication date: Oct-2018
https://doi.org/10.1109/SBGAMES.2018.00022
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents