research-article

The Effects of Hand Representation on Experience and Performance for 3D Interactions in Virtual Reality Games

Authors:

Nicholas Balcomb,

Scott BatemanAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 7, Issue CHI PLAY

Article No.: 420, Pages 1206 - 1233

https://doi.org/10.1145/3611066

Published: 04 October 2023 Publication History

Abstract

In Virtual Reality (VR), natural 3D interactions are performed with hand representations - the visualizations and interactors used for manipulating objects. Hand representations in VR games range from abstract shapes, to graphical versions of input controllers, to realistic human-like hands. Hand representations have been shown to have an important effect on play experience and performance. However, previous work has only considered them for individual 3D interactions or an entire game, giving designers little information about how a representation might perform and be experienced across different 3D interactions (like picking up and rotating objects, or opening a container). In this work, we compare three hand representations across 12 different 3D interactions and in a longer game experience in a study of 45 participants. We find that while representation did not affect performance, representations were overall experienced differently across 3D interactions. Our work provides a deeper understanding for VR game designers about how hand representations can be used to shape play experiences.

References

[1]

Alex Adkins, Lorraine Lin, Aline Normoyle, Ryan Canales, Yuting Ye, and Sophie Jörg. 2021. Evaluating Grasping Visualizations and Control Modes in a VR Game. ACM Transactions on Applied Perception 18, 4 (Oct. 2021 ), 19 : 1-19 : 14. https://doi.org/10.1145/3486582

Digital Library

[2]

Monthir Ali and Rogelio E. Cardona-Rivera. 2020. Comparing Gamepad and Naturally-mapped Controller Efects on Perceived Virtual Reality Experiences. In ACM Symposium on Applied Perception 2020 (SAP '20). Association for Computing Machinery, New York, NY, USA, 1-10. https://doi.org/10.1145/3385955.3407923

Digital Library

[3]

Ferran Argelaguet, Ludovic Hoyet, Michael Trico, and Anatole Lecuyer. 2016. The role of interaction in virtual embodiment: Efects of the virtual hand representation. In 2016 IEEE Virtual Reality (VR). 3-10. https://doi.org/10.1109/ VR. 2016.7504682 ISSN: 2375-5334.

[4]

Lonni Besançon, Paul Issartel, Mehdi Ammi, and Tobias Isenberg. 2017. Mouse, Tactile, and Tangible Input for 3D Manipulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 4727-4740. https://doi.org/10.1145/3025453.3025863

Digital Library

[5]

Matthew Botvinick and Jonathan Cohen. 1998. Rubber hands 'feel' touch that eyes see. Nature 391, 6669 (Feb. 1998 ), 756-756. https://doi.org/10.1038/35784 Number: 6669 Publisher: Nature Publishing Group.

[6]

Ryan Canales, Aline Normoyle, Yu Sun, Yuting Ye, Massimiliano Di Luca, and Sophie Jörg. 2019. Virtual Grasping Feedback and Virtual Hand Ownership. In ACM Symposium on Applied Perception 2019 (SAP '19). Association for Computing Machinery, New York, NY, USA, 1-9. https://doi.org/10.1145/3343036.3343132

Digital Library

[7]

Andreea Dalia Blaga, Maite Frutos-Pascual, Chris Creed, and Ian Williams. 2021. Freehand Grasping: An Analysis of Grasping for Docking Tasks in Virtual Reality. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). 749-758. https://doi.org/10.1109/VR50410. 2021.00102 ISSN: 2642-5254.

[8]

Diane Dewez, Ludovic Hoyet, Anatole Lécuyer, and Ferran Argelaguet Sanz. 2021. Towards Avatar-Friendly 3D Manipulation Techniques: Bridging the Gap Between Sense of Embodiment and Interaction in Virtual Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3411764.3445379

Digital Library

[9]

Paul Dourish. 2004. Where the Action Is: The Foundations of Embodied Interaction (new ed edition ed.). MIT Press, Cambridge, Mass. London.

[10]

Mostafa Elbehery, Florian Weidner, and Wolfgang Broll. 2020. Haptic Space: The Efect of a Rigid Hand Representation on Presence when Interacting with Passive Haptics Controls in VR. In 19th International Conference on Mobile and Ubiquitous Multimedia (MUM 2020 ). Association for Computing Machinery, New York, NY, USA, 245-253. https: //doi.org/10.1145/3428361.3428388

Digital Library

[11]

Martin Feick, Scott Bateman, Anthony Tang, André Miede, and Nicolai Marquardt. 2020. Tangi: Tangible Proxies For Embodied Object Exploration And Manipulation In Virtual Reality. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 195-206. https://doi.org/10.1109/ISMAR50242. 2020.00042 ISSN: 1554-7868.

[12]

Quantic Foundry. 2022. Gamer Motivation Profile. https://apps.quanticfoundry.com/surveys/start/gamerprofile/

[13]

Rebecca Fribourg, Ferran Argelaguet, Anatole Lécuyer, and Ludovic Hoyet. 2020. Avatar and Sense of Embodiment: Studying the Relative Preference Between Appearance, Control and Point of View. IEEE Transactions on Visualization and Computer Graphics 26, 5 (May 2020 ), 2062-2072. https://doi.org/10.1109/TVCG. 2020.2973077 Conference Name: IEEE Transactions on Visualization and Computer Graphics.

[14]

Guilherme Gonçalves, Miguel Melo, Luís Barbosa, José Vasconcelos-Raposo, and Maximino Bessa. 2022. Evaluation of the impact of diferent levels of self-representation and body tracking on the sense of presence and embodiment in immersive VR. Virtual Reality 26, 1 (March 2022 ), 1-14. https://doi.org/10.1007/s10055-021-00530-5

Digital Library

[15]

Koichi Hirota and Kazuyoshi Tagawa. 2016. Interaction with virtual object using deformable hand. In 2016 IEEE Virtual Reality (VR). 49-56. https://doi.org/10.1109/VR. 2016.7504687 ISSN: 2375-5334.

[16]

Ludovic Hoyet, Ferran Argelaguet, Corentin Nicole, and Anatole Lécuyer. 2016. “ Wow! I Have Six Fingers!”: Would You Accept Structural Changes of Your Hand in VR? Frontiers in Robotics and AI 3 ( 2016 ). https://www.frontiersin.org/ article/10.3389/frobt. 2016.00027

[17]

Markus Höll, Markus Oberweger, Clemens Arth, and Vincent Lepetit. 2018. Eficient Physics-Based Implementation for Realistic Hand-Object Interaction in Virtual Reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 175-182. https://doi.org/10.1109/VR. 2018.8448284

[18]

Boris Iglewicz and David C Hoaglin. 1993. Volume 16 : how to detect and handle outliers. Quality Press.

[19]

Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (CHI '97). Association for Computing Machinery, New York, NY, USA, 234-241. https://doi.org/10.1145/258549.258715

Digital Library

[20]

Camille Jeunet, Louis Albert, Ferran Argelaguet, and Anatole Lécuyer. 2018. “ Do You Feel in Control?”: Towards Novel Approaches to Characterise, Manipulate and Measure the Sense of Agency in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics 24, 4 (April 2018 ), 1486-1495. https://doi.org/10.1109/TVCG. 2018.2794598 Conference Name: IEEE Transactions on Visualization and Computer Graphics.

[21]

Konstantina Kilteni, Ilias Bergstrom, and Mel Slater. 2013. Drumming in Immersive Virtual Reality: The Body Shapes the Way We Play. IEEE Transactions on Visualization and Computer Graphics 19, 4 (April 2013 ), 597-605. https://doi.org/10.1109/TVCG. 2013.29 Conference Name: IEEE Transactions on Visualization and Computer Graphics.

[22]

Konstantina Kilteni, Raphaela Groten, and Mel Slater. 2012. The Sense of Embodiment in Virtual Reality. Presence 21, 4 (Nov. 2012 ), 373-387. https://doi.org/10.1162/PRES_a_00124 Conference Name: Presence.

Digital Library

[23]

Joseph J. LaViola Jr., Ernst Kruijf, Ryan P. McMahan, Doug A. Bowman, and Ivan Poupyrev. 2017. 3D User Interfaces Theory and Practice (second ed.). Addison-Wesley Professional.

[24]

Lorraine Lin and Sophie Jörg. 2016. Need a hand? how appearance afects the virtual hand illusion. In Proceedings of the ACM Symposium on Applied Perception (SAP '16). Association for Computing Machinery, New York, NY, USA, 69-76. https://doi.org/10.1145/2931002.2931006

Digital Library

[25]

Christos Lougiakis, Akrivi Katifori, Maria Roussou, and Ioannis-Panagiotis Ioannidis. 2020. Efects of Virtual Hand Representation on Interaction and Embodiment in HMD-based Virtual Environments Using Controllers. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 510-518. https://doi.org/10.1109/VR46266. 2020.00072 ISSN: 2642-5254.

[26]

Jean-Luc Lugrin, Maximilian Ertl, Philipp Krop, Richard Klüpfel, Sebastian Stierstorfer, Bianka Weisz, Maximilian Rück, Johann Schmitt, Nina Schmidt, and Marc Erich Latoschik. 2018. Any “Body” There? Avatar Visibility Efects in a Virtual Reality Game. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 17-24. https: //doi.org/10.1109/VR. 2018.8446229

[27]

Ayano Miyazaki, Hiroko Ota, Kumiko Iwazaki, Mayu Urata, Mamoru Endo, and Takami Yasuda. 2021. Hand Trackingbased VR Ikebana Manipulation. In 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE). 723-726. https: //doi.org/10.1109/GCCE53005. 2021.9621994 ISSN: 2378-8143.

[28]

David Panzoli, Paco Royeres, and Mathéo Fedou. 2019. Hand-based interactions in Virtual Reality: No better feeling than the real thing!. In 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games). 1-2. https://doi.org/10.1109/VS-Games. 2019.8864546 ISSN: 2474-0489.

[29]

Duc-Minh Pham and Wolfgang Stuerzlinger. 2019. Is the Pen Mightier than the Controller? A Comparison of Input Devices for Selection in Virtual and Augmented Reality. In 25th ACM Symposium on Virtual Reality Software and Technology (VRST '19). Association for Computing Machinery, New York, NY, USA, 1-11. https://doi.org/10.1145/ 3359996.3364264

Digital Library

[30]

Aylen Ricca, Amine Chellali, and Samir Otrnane. 2021. The influence of hand visualization in tool-based motor-skills training, a longitudinal study. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). 103-112. https://doi.org/10.1109/ VR50410. 2021.00031 ISSN: 2642-5254.

[31]

Daniel Roth and Marc Erich Latoschik. 2020. Construction of the Virtual Embodiment Questionnaire (VEQ). IEEE Transactions on Visualization and Computer Graphics 26, 12 (Dec. 2020 ), 3546-3556. https://doi.org/10.1109/TVCG. 2020. 3023603 Conference Name: IEEE Transactions on Visualization and Computer Graphics.

[32]

Richard M. Ryan, C. Scott Rigby, and Andrew Przybylski. 2006. The Motivational Pull of Video Games: A SelfDetermination Theory Approach. Motivation and Emotion 30 ( 2006 ), 347-365. https://doi.org/10.1007/s11031-006-9051-8

[33]

Valentin Schwind, Pascal Knierim, Lewis Chuang, and Niels Henze. 2017. "Where's Pinky?": The Efects of a Reduced Number of Fingers in Virtual Reality. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). Association for Computing Machinery, New York, NY, USA, 507-515. https://doi.org/10.1145/3116595.3116596

Digital Library

[34]

Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017. "These are not my hands!": Efect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1577-1582. http://doi.org/10.1145/3025453.3025602

Digital Library

[35]

Valentin Schwind, Sven Mayer, Alexandre Comeau-Vermeersch, Robin Schweigert, and Niels Henze. 2018. Up to the Finger Tip: The Efect of Avatars on Mid-Air Pointing Accuracy in Virtual Reality. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). Association for Computing Machinery, New York, NY, USA, 477-488. https://doi.org/10.1145/3242671.3242675

Digital Library

[36]

Sofia Seinfeld, Tiare Feuchtner, Antonella Maselli, and Jörg Müller. 2021. User Representations in Human-Computer Interaction. Human-Computer Interaction 36, 5-6 (Oct. 2021 ), 400-438. https://doi.org/10.1080/07370024. 2020.1724790 Publisher: Taylor & Francis_eprint: https://doi.org/10.1080/07370024. 2020. 1724790.

[37]

Sofia Seinfeld, Tiare Feuchtner, Johannes Pinzek, and Jörg Müller. 2020. Impact of Information Placement and User Representations in VR on Performance and Embodiment. Vol. PP. https://doi.org/10.1109/TVCG. 2020.3021342 Journal Abbreviation: IEEE transactions on visualization and computer graphics Publication Title: IEEE transactions on visualization and computer graphics.

[38]

Paul Skalski, Ron Tamborini, Ashleigh Shelton, Michael Buncher, and Pete Lindmark. 2011. Mapping the road to fun: Natural video game controllers, presence, and game enjoyment. New Media & Society 13, 2 (March 2011 ), 224-242. https://doi.org/10.1177/1461444810370949 Publisher: SAGE Publications.

[39]

Tanh Quang Tran, HyunJu Shin, Wolfgang Stuerzlinger, and JungHyun Han. 2017. Efects of virtual arm representations on interaction in virtual environments. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (VRST '17). Association for Computing Machinery, New York, NY, USA, 1-9. https://doi.org/10.1145/3139131. 3139149

Digital Library

[40]

Michiel Van Veldhuizen and Xubo Yang. 2021. The Efect of Semi-Transparent and Interpenetrable Hands on Object Manipulation in Virtual Reality. IEEE Access 9 ( 2021 ), 17572-17583. https://doi.org/10.1109/ACCESS. 2021.3050881 Conference Name: IEEE Access.

[41]

Huagen Wan, Yang Luo, Shuming Gao, and Qunsheng Peng. 2004. Realistic virtual hand modeling with applications for virtual grasping. In Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry (VRCAI '04). Association for Computing Machinery, New York, NY, USA, 81-87. https: //doi.org/10.1145/1044588.1044603

Digital Library

[42]

Nick Yee. 2006. Motivations for Play in Online Games. CyberPsychology and Behavior 9, 6 (Dec. 2006 ), 772-775. https://doi.org/10.1089/cpb. 2006. 9. 772

[43]

Nick Yee and Jeremy Bailenson. 2007. The Proteus Efect: The Efect of Transformed Self-Representation on Behavior. Human Communication Research 33, 3 ( 2007 ), 271-290. https://doi.org/10.1111/j.1468-2958. 2007. 00299.x arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1468-2958. 2007. 00299.x

[44]

[G1] Beat Games. 2019. Beat Saber. Game [PCVR].

[45]

[G2] Bethesda Game Studios. 2017. Fallout 4: VR. Game [PCVR].

[46]

[G3] Bethesda Game Studios. 2018. The Elder Scrolls V: Skyrim VR. Game [PCVR].

[47]

[G4] BlackTale Games. 2020. Tales of Glory. Game [PCVR].

[48]

[G5] Boundless Dynamics, LLC. 2017. VTOL VR. Game [PCVR].

[49]

[G6] Cloudhead Games. 2019. Pistol Whip. Game [PCVR].

[50]

[G7] Downpour Interactive. 2016. Onward. Game [PCVR].

[51]

[G8] Free Lives. 2019. Gorn. Game [PCVR].

[52]

[G9] id Software. 2017. Doom VFR. Game [PCVR].

[53]

[G10] JoyWay Ltd. 2021. STRIDE. Game [PCVR].

[54]

[G11] jrbudda and Techjar. 2016. Vivecraft: Virtual Reality Minecraft for SteamVR. Game Mod [PCVR].

[55]

[G12] Just Add Water. 2021. Sniper Elite VR. Game [PCVR].

[56]

[G13] Mighty Coconut. 2021. Walkabout Mini Golf VR. Game [PCVR].

[57]

[G14] Mojang. 2016. Minecraft VR for Windows. Game [Oculus/PCVR].

[58]

[G15] Motive Studio. 2020. STAR WARS: Squadrons. Game [PCVR].

[59]

[G16] Owlchemy Labs. 2016. Job Simulator. Game [PCVR].

[60]

[G17] Polyarc. 2018. Moss. Game [PSVR].

[61]

[G18] Ready At Dawn. 2017. Lone Echo. Game [Oculus Rift].

[62]

[G19] RUST LTD. 2016. Hot Dogs, Horseshoes & Hand Grenades. Game [PCVR].

[63]

[G20] Skydance Interactive. 2020. The Walking Dead: Saints & Sinners. Game [PCVR].

[64]

[G21] Steel Crate Games. 2015. Keep Talking and Nobody Explodes. Game [PCVR].

[65]

[G22] Stress Level Zero. 2019. Boneworks. Game [PCVR].

[66]

[G23] Superhot Team. 2016. Superhot VR. Game [PCVR].

[67]

[G24] Urban Wolf Games Inc. 2021. Legendary Tales. Game [PCVR].

[68]

[G25] Valve. 2020. Half-Life: Alyx. Game [PCVR].

[69]

[G26] Vertigo Games and Jaywalkers Interactive. 2016. Arizona Sunshine. Game [PCVR].

[70]

[G27] WarpFrog. 2018. Blade and Sorcery. Game [PCVR].

Cited By

Hwang ASiy JShelby RLentz A(2024)In Whose Voice?: Examining AI Agent Representation of People in Social Interaction through Generative SpeechProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661555(224-245)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661555

Index Terms

The Effects of Hand Representation on Experience and Performance for 3D Interactions in Virtual Reality Games
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Virtual reality
    2. Interaction techniques
  2. Interaction design

Recommendations

Investigating Player Experience in Virtual Reality Games via Remote Experimentation
CHI PLAY '21: Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play

This research explores player experience of virtual reality (VR) games through two stages of study. In both stages, we employed the Player Experience Inventory (PXI), a validated tool designed to evaluate player experience. In Stage 1, player experience ...
Comparing Player Experience in Video Games Played in Virtual Reality or on Desktop Displays: Immersion, Flow, and Positive Emotions
CHI PLAY '19 Extended Abstracts: Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts

Scientific knowledge of the differences between video games played in virtual reality and on desktop displays in terms of player experience is still limited. Therefore, this study aimed to explore differences in immersion, flow, positive emotions, and ...
Gaming in Virtual Reality: What Changes in Terms of Usability, Emotional Response and Sense of Presence Compared to Non-Immersive Video Games?

Background. Virtual reality can provide innovative gaming experiences for present and future game players. However, scientific knowledge is still limited about differences between player’s experience in video games played in immersive modalities and games ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 7, Issue CHI PLAY

November 2023

1360 pages

EISSN:2573-0142

DOI:10.1145/3554313

Editor:
Jeffrey Nichols
Apple, USA

Issue’s Table of Contents

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2023

Published in PACMHCI Volume 7, Issue CHI PLAY

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
272
Total Downloads

Downloads (Last 12 months)174
Downloads (Last 6 weeks)26

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hwang ASiy JShelby RLentz A(2024)In Whose Voice?: Examining AI Agent Representation of People in Social Interaction through Generative SpeechProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661555(224-245)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661555

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents