opinion

Free access

3D Audio: The Next Step in Realism for Gaming: Benefits and pitfalls of 3D audio in games from a creative content and technical implementation point of view

Authors:

Martin Rieger,

Dominik ZinglerAuthors Info & Claims

ACM Games: Research and Practice, Volume 1, Issue 1

Article No.: 12, Pages 1 - 3

https://doi.org/10.1145/3583059

Published: 12 March 2023 Publication History

PDF eReader

Abstract

When using 3D audio in games, the goal should be to create an immersive, believable auditory experience. To achieve this on the technical side of things, game developers must combine the appropriate hardware with sophisticated software algorithms and sound propagation techniques. Regarding content, creators need to distinguish between 3D sounds inside the scene and sounds with a narrative function that are non-diegetic. This article looks behind the curtains of the spatial audio hype and lets you know what you're in for and what is coming in the future.

1 Introduction

There can be little doubt that 3D audio can greatly enhance the gaming experience when designed well. Being able to hear sounds from all directions can make for a considerably more immersive and exciting game. However, there are also some pitfalls to consider when using 3D audio in games. In this article we will discuss the state of the art and future potential of this so-called “spatial Audio”.

2 Make Everything 3d – or Not!?

Spatial Audio is utilized within digital gaming spaces to help players experience sound like in real-life – sounds happening around them, being immersed inside the experience, maybe even making them forget to wear headphones. This phenomenon, called externalization, is meant to help players discern which sounds are part of the scene.

While 3D audio is a fantastic tool in the game developer's toolkit, if it is not used carefully, 3D audio can become distracting. For example, mono playback is still favored for narration and stereo is preferred for music. The reason is that when wearing headphones, you'll localize the sound within your head, not around you. This is called inside head locatedness (IHL). This is where the sounds waves traveling from the left and right loudspeakers meet – inside of our head. As we have become used to this, with the introduction of 3D audio it becomes clearer that this habit can become a problem.

The key to making spatial audio most effective is to ensure the player can immediately discern the context of the sounds they are experiencing. For example, if the sound of a voice can be localized, by moving through the scene, the player would know that the sound is part of the scene (3D sound). Meaning, there must be another character close by talking. If the sound cannot be localized when moving around the scene (mono sound), so it's independent from any movement, the player would understand that the voice does not represent a character in the scene. Therefore, it must be a form of narration, giving you information about the scene, while not being visually represented.

3 The Future of Audio

3.1 Head-Tracking

So, what's next? Gamers are used to looking at a screen in front of them. But in our daily lives, our head makes continuous micro-movements which help us to orient ourselves in our environment or when watching a screen. This is where head-tracking introduces the next evolution of 3D audio. Slight movements help to distinguish sounds coming from the front with sounds from the back.

With this new infrastructure of hardware, there will be a new market for games that put sound first. They could be audio-only or have auditory cues where your ears help your eyes to unlock the next achievement. This will create a new way to immerse yourself in a game. With most games, you just watch, facing to the front. Now with head-tracking, you are free to move your head on multiple axes. Virtual reality (VR) experiences are already taking advantage of the physical body movement. But what many of these virtual worlds are missing is true realism.

Apple is already building spatial audio technology and head-tracking into their AirPods! So, there are millions of devices on the market already. Gaming headsets are also starting to implement gyroscopes or webcam trackers as add-ons.

3.2 Increased Realism

In addition to new hardware, there is a growing interest in increased realism and higher fidelity in audio, especially in games. This strong focus on graphic fidelity left few resources for research and development of more realistic in-game audio.

With increasing diminishing returns of investments in graphic fidelity, more and more studios and developers are getting back to game audio, seeing its enormous potential to improve the player's experience. Topics like 3D audio rendering, realistic room simulation, and sound propagation are getting more attention to finally create fully realistic audio in games.

One of the major pitfalls in the drive to reach a higher level of realism is that true acoustic reality is impossible and often undesirable to achieve. The true goal should be an authentic, natural-feeling, high-fidelity psychoacoustics based auditory world.

Audio designers will always have to creatively adapt a simulated environment to a game's needs. It is similar to sound attenuation being rarely based upon a physically correct simulation of the decrease in sound pressure level and perceived loudness. Next Generation Audio features the need to be carefully integrated into a game's content with an extensive toolset that enables designers to fully achieve their creative visions.

3.3 Real-time Room Simulation

The future of audio in games will also be improvements in real-time room simulation. While simulating sound obstruction in video games with simple ray-casting methods have been common for years, more sophisticated sound occlusion, absorption, transmission, and propagation methods operating with ray tracing operations are still uncommon.

Now that more people have better access to high-performance hardware in the form of current-gen consoles or cloud computing via streaming, these technologies will start to seem more and more viable. But even in their near-future state, even the most advanced room simulation technology would still have scope for improvement, such as the creation of more sophisticated and optimized early reflection systems that are using all the advantages of 3D audio. The exact positioning of the first order early reflections that are not only spatialized but also each have their own occlusion values calculated will provide a more natural sounding expansion of audio waves in a simulated environment.

4 Conclusion

The games industry is always on the lookout for new ways to improve the gaming experience, and 3D audio is one way that seems to hold a lot of promise. However, with the right combination of technologies, 3D audio can not only improve the gaming experience but create new ways to play using head-tracking. The computer graphics industry has been striving for realism for years. Now it is up to game audio to lead the way to even higher fidelity, and 3D audio will only be the first step in achieving this goal.

Cited By

View all

Bosman ITordiffe RHuang TBuruk OJørgensen KHamari J(2024)SoundTrek: A Virtual Reality Audio Perspective-Switching GameCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678824(352-358)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3665463.3678824
Jennes IBlanckaert EVan den Broeck W(2023)Immersion or Disruption?Proceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596151(129-139)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596151

Index Terms

3D Audio: The Next Step in Realism for Gaming: Benefits and pitfalls of 3D audio in games from a creative content and technical implementation point of view

Recommendations

A framework for immersive spatial audio performance
NIME '06: Proceedings of the 2006 conference on New interfaces for musical expression

Traditional uses of virtual audio environments tend to focus on perceptually accurate acoustic representations. Though spatialization of sound sources is important, it is necessary to leverage control of the sonic representation when considering musical ...
Audio games: new perspectives on game audio
ACE '04: Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology

This paper discusses the design of audio games, a quite new computer game category that originates from games for players with visual impairments as well as from mainstream music games. In the TiM project (Tactile Interactive Multimedia), SITREC ...
Generative Audio and Real-Time Soundtrack Synthesis in Gaming Environments: An exploration of how dynamically rendered soundtracks can introduce new artistic sound design opportunities and enhance the immersion of interactive audio spaces.
OzCHI '20: Proceedings of the 32nd Australian Conference on Human-Computer Interaction

An important yet oft-overlooked front in the scope of interactive media, audio technologies have remained relatively stagnant compared to groundbreaking advancements made in fields such as visual fidelity and virtual reality. This paper explores the use ...

Comments

Information & Contributors

Information

Published In

cover image Games: Research and Practice

Games: Research and Practice Volume 1, Issue 1

March 2023

103 pages

EISSN:2832-5516

DOI:10.1145/3581756

Editors:
Sebastian Deterding
Imperial College London, U.K.
,
Kenny Mitchell
Roblox/Edinburgh Napier U, USA

Issue’s Table of Contents

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 March 2023

Accepted: 01 February 2023

Received: 30 November 2022

Published in GAMES Volume 1, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Opinion

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
1,722
Total Downloads

Downloads (Last 12 months)1,072
Downloads (Last 6 weeks)150

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Bosman ITordiffe RHuang TBuruk OJørgensen KHamari J(2024)SoundTrek: A Virtual Reality Audio Perspective-Switching GameCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678824(352-358)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3665463.3678824
Jennes IBlanckaert EVan den Broeck W(2023)Immersion or Disruption?Proceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596151(129-139)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596151

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

1 Introduction

2 Make Everything 3d – or Not!?

3 The Future of Audio

3.1 Head-Tracking

3.2 Increased Realism

3.3 Real-time Room Simulation

4 Conclusion

Cited By

Index Terms

Recommendations

A framework for immersive spatial audio performance

Audio games: new perspectives on game audio

Generative Audio and Real-Time Soundtrack Synthesis in Gaming Environments: An exploration of how dynamically rendered soundtracks can introduce new artistic sound design opportunities and enhance the immersion of interactive audio spaces.

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations