SoundTrek: A Virtual Reality Audio Perspective-Switching Game
Authors: 
Isak de Villiers Bosman, Ross Tordiffe, Tianyou 'Jackie' Huang, 
Oğuz 'Oz' Buruk, Kristine Jørgensen, Juho HamariAuthors Info & Claims
Isak de Villiers Bosman, Ross Tordiffe, Tianyou 'Jackie' Huang, Oğuz 'Oz' Buruk, Kristine Jørgensen, Juho HamariAuthors Info & ClaimsPages 352 - 358
Abstract
Virtual reality technologies allow new ways of creating a sense of space not only through sight but also through sound. We present SoundTrek, a game about using listening perspective for navigating through environments in VR. In SoundTrek, the player needs to switch between listening to spatialized sounds from the first- and third-person perspectives to navigate through obstacles. SoundTrek also employs sound design strategies, such as spatialized soundtracks and near-field audio to aid spatial navigation in the absence of visual cues and to create unique aesthetic experiences through sound. We present new ways of utilizing sound in virtual reality game design that highlights design considerations for audio-focused gameplay and diversifies the opportunities for creating rich, interactive soundscapes.
Supplemental Material
ZIP File - Gameplay video: SoundTrek
A short video demonstrating the gameplay for SoundTrek
- Download
- 26.83 MB
ZIP File - SoundTrek demo
Playable demo of SoundTrek
- Download
- 89.22 MB
References
[1]
Raul Altosaar, Adam Tindale, and Judith Doyle. 2019. Physically Colliding with Music: Full-body Interactions with an Audio-only Virtual Reality Interface. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction(TEI ’19). Association for Computing Machinery, New York, NY, USA, 553–557. https://doi.org/10.1145/3294109.3301256
[2]
Neil Barbour. 2020. Top 10 VR Games By Revenue. https://www.spglobal.com/marketintelligence/en/news-insights/blog/top-10-vr-games-by-revenue
[3]
Isak de Villiers Bosman, Oğuz ‘Oz’ Buruk, Kristine Jørgensen, and Juho Hamari. 2023. The effect of audio on the experience in virtual reality: a scoping review. Behaviour & Information Technology 43, 1 (Jan. 2023), 165–199. https://doi.org/10.1080/0144929X.2022.2158371 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/0144929X.2022.2158371.
[4]
Paul Cairns, Christopher Power, Mark Barlet, and Greg Haynes. 2019. Future design of accessibility in games: A design vocabulary. International Journal of Human-Computer Studies 131 (Nov. 2019), 64–71. https://doi.org/10.1016/j.ijhcs.2019.06.010
[5]
Gordon Calleja. 2007. Digital Game Involvement: A Conceptual Model. Games and Culture 2, 3 (July 2007), 236–260. https://doi.org/10.1177/1555412007306206 Publisher: SAGE Publications.
[6]
Erik Carlson. 2007. Audio Maps. Ph. D. Dissertation. Linköping University, The Institute of Technology., Linköping, Sweden. https://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-96206
[7]
Michel Chion. 1994. Audio-vision: Sound on Screen. Columbia University Press. Google-Books-ID: BBs4Arfm98oC.
[8]
Marko Ciciliani. 2020. Virtual 3D environments as composition and performance spaces*. Journal of New Music Research 49, 1 (Jan. 2020), 104–113. https://doi.org/10.1080/09298215.2019.1703013 Publisher: Routledge _eprint: https://doi.org/10.1080/09298215.2019.1703013.
[9]
Karen Collins. 2013. Sonic Subjectivity and Auditory Perspective in Ratatouille. Animation 8, 3 (Nov. 2013), 283–299. https://doi.org/10.1177/1746847713507164
[10]
Karen Collins. 2020. Game Sound: Reverberations. Journal of Sound and Music in Games 1, 1 (Jan. 2020), 100–102. https://doi.org/10.1525/jsmg.2020.1.1.100 Publisher: University of California Press.
[11]
Karen Collins and Bill Kapralos. 2012. Beyond the screen: what we can Learn about Game Design from Audio-Based Games. In Proceedings of the Computer Games Multimedia and Allied Technology (CGAT 2012) Conference. Bali, Indonesia, 5.
[12]
Dixuan Cui and Christos Mousas. 2023. Evaluating the Sense of Embodiment through Out-of-Body Experience and Tactile Feedback. In Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry(VRCAI ’22). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3574131.3574456
[13]
Frédérique de Vignemont. 2011. Embodiment, ownership and disownership. Consciousness and Cognition 20, 1 (March 2011), 82–93. https://doi.org/10.1016/j.concog.2010.09.004
[14]
Henrique Galvan Debarba, Sidney Bovet, Roy Salomon, Olaf Blanke, Bruno Herbelin, and Ronan Boulic. 2017. Characterizing first and third person viewpoints and their alternation for embodied interaction in virtual reality. PLOS ONE 12, 12 (Dec. 2017), e0190109. https://doi.org/10.1371/journal.pone.0190109 Publisher: Public Library of Science.
[15]
Delphine Devallez. 2009. Auditory perspective: perception, rendering, and applications. Doctoral thesis. Universit‘a di Verona, Verona, Italy. https://iris.univr.it/handle/11562/337377
[16]
Johnny Friberg and Dan Gärdenfors. 2004. Audio games: new perspectives on game audio. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology(ACE ’04). Association for Computing Machinery, New York, NY, USA, 148–154. https://doi.org/10.1145/1067343.1067361
[17]
Tom Alexander Garner. 2013. Game Sound from Behind the Sofa: An Exploration into the Fear Potential of Sound & Psychophysiological Approaches to Audio-centric, Adaptive Gameplay. (2013).
[18]
Geoffrey Gorisse, Olivier Christmann, Etienne Armand Amato, and Simon Richir. 2017. First- and Third-Person Perspectives in Immersive Virtual Environments: Presence and Performance Analysis of Embodied Users. Frontiers in Robotics and AI 4 (2017). https://www.frontiersin.org/articles/10.3389/frobt.2017.00033
[19]
D. Griesinger. 2018. Localization, loudness, and proximity. In Proceedings of the Institute of Acoustics, Vol. 40. Hamburg, 294–302. ISSN: 1478-6095.
[20]
João Guerreiro, Yujin Kim, Rodrigo Nogueira, SeungA Chung, André Rodrigues, and Uran Oh. 2023. The Design Space of the Auditory Representation of Objects and Their Behaviours in Virtual Reality for Blind People. IEEE Transactions on Visualization and Computer Graphics 29, 5 (May 2023), 2763–2773. https://doi.org/10.1109/TVCG.2023.3247094
[21]
Douglas Henderson. 2022. Gespräch mit dem amerikanischen Künstler Douglas Henderson. https://www.youtube.com/watch?v=y6Fsuvax_jY
[22]
Matthias Hoppe, Andrea Baumann, Patrick Chofor Tamunjoh, Tonja-Katrin Machulla, Paweł W. Woźniak, Albrecht Schmidt, and Robin Welsch. 2022. There Is No First- or Third-Person View in Virtual Reality: Understanding the Perspective Continuum. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3491102.3517447
[23]
Claudia Jenny and Christoph Reuter. 2021. Can I trust my ears in VR? Head-related transfer functions and valuation methods with descriptive attributes in virtual reality. International Journal of Virtual Reality 21, 2 (Oct. 2021), 29–43. https://doi.org/10.20870/IJVR.2021.21.2.4831 Number: 2.
[24]
Kristine Jørgensen. 2007. On transdiegetic sounds in computer games. Northern Lights: Film & Media Studies Yearbook 5, 1 (Sept. 2007), 105–117. https://doi.org/10.1386/nl.5.1.105_1
[25]
Kristine Jørgensen. 2013. Gameworld Interfaces. MIT Press, Cambridge, Massachusetts. Google-Books-ID: sGVWAgAAQBAJ.
[26]
Kristine Jørgensen. 2017. Left in the dark: playing computer games with the sound turned off. In From Pac-Man to Pop Music (1 ed.), Karen Collins (Ed.). Routledge, 163–176. https://doi.org/10.4324/9781351217743-12
[27]
Haven Kim, Jaeran Choi, Young Yim Doh, and Juhan Nam. 2022. The Melody of the Mysterious Stones: A VR Mindfulness Game Using Sound Spatialization. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems(CHI EA ’22). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3491101.3516490
[28]
Merriam-Webster. 2024. Definition of PERSPECTIVE. https://www.merriam-webster.com/dictionary/perspective
[29]
Raphaël Millière, Robin L. Carhart-Harris, Leor Roseman, Fynn-Mathis Trautwein, and Aviva Berkovich-Ohana. 2018. Psychedelics, Meditation, and Self-Consciousness. Frontiers in Psychology 9 (2018). https://www.frontiersin.org/articles/10.3389/fpsyg.2018.01475
[30]
Diego Monteiro, Hai-Ning Liang, Wenge Xu, Marvin Brucker, Vijayakumar Nanjappan, and Yong Yue. 2018. Evaluating enjoyment, presence, and emulator sickness in VR games based on first- and third- person viewing perspectives. Computer Animation and Virtual Worlds 29, 3-4 (2018), e1830. https://doi.org/10.1002/cav.1830 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/cav.1830.
[31]
Playful Studios. 2024. New Super Lucky’s Tale. https://playfulstudios.com/new-super-luckys-tale/
[32]
Polyarc Games. 2024. Moss. https://www.polyarcgames.com/games/moss
[33]
[33] Benjamin Kym Probert. 2020. Video Game Acoustics: Perception-Based Sound Design for Interactive Virtual Spaces Submitted. Thesis. https://digital.library.adelaide.edu.au/dspace/handle/2440/129585Accepted: 2021-01-18T02:31:29Z.
[34]
Renato Alexandre Ribeiro, Inês Gonçalves, Manuel Piçarra, Letícia Seixas Pereira, Carlos Duarte, André Rodrigues, and João Guerreiro. 2024. Investigating Virtual Reality Locomotion Techniques with Blind People. In Proceedings of the CHI Conference on Human Factors in Computing Systems(CHI ’24). Association for Computing Machinery, New York, NY, USA, 1–17. https://doi.org/10.1145/3613904.3642088
[35]
Martin Rieger and Dominik Zingler. 2023. 3D Audio: The Next Step in Realism for Gaming: Benefits and pitfalls of 3D audio in games from a creative content and technical implementation point of view. Games: Research and Practice 1, 1 (March 2023), 12:1–12:3. https://doi.org/10.1145/3583059
[36]
Paul Roquet. 2009. Ambient Landscapes from Brian Eno to Tetsu Inoue. Journal of Popular Music Studies 21, 4 (2009), 364–383. https://doi.org/10.1111/j.1533-1598.2009.01208.x _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1533-1598.2009.01208.x.
[37]
Stefania Serafin. 2020. Sonic Interactions in Multimodal Virtual Environments. array. the journal of the ICMA (Oct. 2020), 16–18. https://doi.org/10.25370/array.v20152517
[38]
Jolie H. K. Smets and Erik D. van der Spek. 2021. That Sound’s Juicy! Exploring Juicy Audio Effects in Video Games. In Entertainment Computing – ICEC 2021, Jannicke Baalsrud Hauge, Jorge C. S. Cardoso, Licínio Roque, and Pedro A. Gonzalez-Calero (Eds.). Springer International Publishing, Cham, 319–335. https://doi.org/10.1007/978-3-030-89394-1_24
[39]
Pallavi Sodhi, Audrey Girouard, and David Thue. 2023. Accessible Play: Towards Designing a Framework for Customizable Accessibility in Games. In Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play(CHI PLAY Companion ’23). Association for Computing Machinery, New York, NY, USA, 49–55. https://doi.org/10.1145/3573382.3616075
[40]
Tim Summers, James Cook, Will Famer, Elisa Raffaella Ferrè, Lucy Harrison, Rich Hemming, Andra Ivănescu, Luke Reed, Flossie Roberts, Richard Stevens, Stephen Tatlow, and Laryssa Whittaker. 2021. Music and Sound in Virtual/Augmented Realities — Questions, Challenges and Approaches: A Multidisciplinary Roundtable. Journal of Sound and Music in Games 2, 2 (April 2021), 63–83. https://doi.org/10.1525/jsmg.2021.2.2.63 Publisher: University of California Press.
[41]
Jaime Sánchez and Mauricio Lumbreras. 1999. Virtual Environment Interaction Through 3D Audio by Blind Children. CyberPsychology & Behavior 2, 2 (April 1999), 101–111. https://doi.org/10.1089/cpb.1999.2.101 Publisher: Mary Ann Liebert, Inc., publishers.
[42]
Isabella van Elferen. 2020. Ludomusicology and the New Drastic. Journal of Sound and Music in Games 1, 1 (Jan. 2020), 103–112. https://doi.org/10.1525/jsmg.2020.1.1.103 Publisher: University of California Press.
[43]
Bruce N. Walker and Jeffrey Lindsay. 2006. Navigation Performance With a Virtual Auditory Display: Effects of Beacon Sound, Capture Radius, and Practice. Human Factors 48, 2 (June 2006), 265–278. https://doi.org/10.1518/001872006777724507 Publisher: SAGE Publications Inc.
[44]
Andrea Stevenson Won and Shuo Zhou. 2024. Effects of first vs. third-person perspective and self- versus other-avatars on user movements in virtual reality. International Journal of Human-Computer Studies 187 (July 2024), 103259. https://doi.org/10.1016/j.ijhcs.2024.103259
[45]
Wenjie Wu and Stefan Rank. 2015. Responsive Environmental Diegetic Audio Feedback for Hand Gestures in Audio-Only Games. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’15). Association for Computing Machinery, New York, NY, USA, 739–744. https://doi.org/10.1145/2793107.2810336
[46]
Anıl Çamcı and Rob Hamilton. 2020. Audio-first VR: New perspectives on musical experiences in virtual environments. Journal of New Music Research 49, 1 (Jan. 2020), 1–7. https://doi.org/10.1080/09298215.2019.1707234 Publisher: Routledge _eprint: https://doi.org/10.1080/09298215.2019.1707234.
Index Terms
- SoundTrek: A Virtual Reality Audio Perspective-Switching Game
Recommendations
Keeping your ears on the virtual ground: Expert insights on virtual reality audio techniques and challenges
AM '24: Proceedings of the 19th International Audio Mostly Conference: Explorations in Sonic CulturesVirtual reality (VR) audio practitioners navigate interdisciplinary spatial design decisions alongside complex interactive audio systems, resulting in novel solutions to creating immersive experiences in VR environments. Previous work has attempted to ...
A mobile augmented reality audio system with binaural microphones
IwS '11: Proceedings of Interacting with Sound Workshop: Exploring Context-Aware, Local and Social Audio ApplicationsThis paper presents a microphone-hear-through augmented reality hardware system, consisting of insert headphones with binaural microphones, i.e., a microphone at each ear, together with a separate mixer and equalizer unit. The equalizer compensates for ...
Designing Games for Presence in Consumer Virtual Reality
SIGMIS-CPR '16: Proceedings of the 2016 ACM SIGMIS Conference on Computers and People ResearchWith virtual reality technologies entering the consumer market this year, it is the task of those producing content for the virtual reality platform to ensure that users have an experience that lives up to expectations. Virtual reality comes with its ...
Comments
Information & Contributors
Information
Published In
October 2024
500 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 October 2024
Check for updates
Badges
Best Artifact
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Academy of Finland Flagship Programme
Conference
CHI PLAY '24
Sponsor:
CHI PLAY '24: The Annual Symposium on Computer-Human Interaction in Play
October 14 - 17, 2024
Tampere, Finland
Acceptance Rates
Overall Acceptance Rate 421 of 1,386 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 100Total Downloads
- Downloads (Last 12 months)100
- Downloads (Last 6 weeks)49
Reflects downloads up to 23 Dec 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in