research-article

Hearing Is Believing: Synthesizing Spatial Audio from Everyday Objects to Users

Authors:

Gábor SörösAuthors Info & Claims

AH2019: Proceedings of the 10th Augmented Human International Conference 2019

Article No.: 28, Pages 1 - 9

https://doi.org/10.1145/3311823.3311872

Published: 11 March 2019 Publication History

Abstract

The ubiquity of wearable audio devices and the importance of the auditory sense imply great potential for audio augmented reality. In this work, we propose a concept and a prototype of synthesizing spatial sounds from arbitrary real objects to users in everyday interactions, whereby all sounds are rendered directly by the user's own ear pods instead of loudspeakers on the objects. The proposed system tracks the user and the objects in real time, creates a simplified model of the environment, and generates realistic 3D audio effects. We thoroughly evaluate the usability and the usefulness of such a system based on a user study with 21 participants. We also investigate how an acoustic environment model improves the sense of engagement of the rendered 3D sounds.

References

[1]

Benjamin B. Bederson. 1995. Audio Augmented Reality: A Prototype Automated Tour Guide. In Conference Companion on Human Factors in Computing Systems (CHI'95).

Digital Library

[2]

Simon Blessenohl, Cecily Morrison, Antonio Criminisi, and Jamie Shotton. 2015. Improving Indoor mobility of The Visually Impaired with Depth-based Spatial Sound. In Proceedings of The IEEE International Conference on Computer Vision Workshops (ICCV'15). 26--34.

Digital Library

[3]

Michele Geronazzo, Erik Sikström, Jari Kleimola, Federico Avanzini, and Stefania Serafin. 2018. The Impact of An Accurate Vertical Localization with HRTFs on Short Explorations of Immersive Virtual Reality Scenarios. In International Symposium on Mixed and Augmented Reality (ISMAR '18).

[4]

Florian Heller and Jan Borchers. 2014. AudioTorch: Using A Smartphone as Directional Microphone in Virtual Audio Spaces. In Proceedings of The 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI'14).

Digital Library

[5]

Florian Heller, Jayan Jevanesan, Pascal Dietrich, and Jan Borchers. 2016. Where Are We?: Evaluating The Current Rendering Fidelity of Mobile Audio Augmented Reality Systems. In Proceedings of The 18th International Conference on HumanComputer Interaction with Mobile Devices and Services (MobileHCI'16).

Digital Library

[6]

Florian Heller and Johannes Schöning. 2018. NavigaTone: Seamlessly Embedding Navigation Cues in Mobile Music Listening. In Proceedings of The 2018 CHI Conference on Human Factors in Computing Systems (CHI'18). 637.

Digital Library

[7]

Kent Lyons, Maribeth Gandy, and Thad Starner. 2000. Guided by Voices: An Audio Augmented Reality System. In Conference on Auditory Display (ICAD'00).

[8]

Jörg Müller, Matthias Geier, Christina Dicke, and Sascha Spors. 2014. The Boom-Room: Mid-air Direct Interaction with Virtual Sound Sources. In Proceedings of The SIGCHI Conference on Human Factors in Computing Systems (CHI'14). 247--256.

Digital Library

[9]

Austin Roorda and David R Williams. 1999. The Arrangement of The Three Cone Classes in The Living Human Eye. Nature 397, 6719 (1999), 520.

[10]

Spencer Russell, Gershon Dublon, and Joseph A Paradiso. 2016. HearThere: Networked Sensory ProsThetics through Auditory Augmented Reality. In Proceedings of The 7th Augmented Human International Conference (AH'16). 20.

Digital Library

[11]

Carl Schissler, Christian Loftin, and Dinesh Manocha. 2018. Acoustic classification and optimization for multi-modal rendering of real-world scenes. IEEE transactions on visualization and computer graphics 24, 3 (2018), 1246--1259.

[12]

Carl Schissler and Dinesh Manocha. 2017. Interactive sound propagation and rendering for large multi-source scenes. ACM Transactions on Graphics 36, 1 (2017), 2.

Digital Library

[13]

Eldon Schoop, James Smith, and Bjoern Hartmann. 2018. HindSight: Enhancing Spatial Awareness by Sonifying Detected Objects in Real-Time 360-Degree Video. In Proceedings of The 2018 SIGCHI Conference on Human Factors in Computing Systems (CHI'18). 143.

Digital Library

[14]

Jaka Sodnik, Saso Tomazic, Raphael Grasset, Andreas Duenser, and Mark Billinghurst. 2006. Spatial Sound Localization in An Augmented Reality Environment. In Proceedings of The 18th Australia Conference on Computer-human Interaction: Design: Activities, Artefacts and Environments (OZCHI'06). 111--118.

Digital Library

[15]

Venkataraman Sundareswaran, Kenneth Wang, Steven Chen, Reinhold Behringer, Joshua McGee, Clement Tam, and Pavel Zahorik. 2003. 3D audio augmented reality: implementation and experiments. In Proceedings of The 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR'03). IEEE Computer Society, 296.

Digital Library

[16]

Titus JJ Tang and Wai Ho Li. 2014. An Assistive Eyewear Prototype That Interactively Converts 3D Object Locations into Spatial Audio. In Proceedings of The 2014 ACM International Symposium on Wearable Computers (ISWC'14). 119--126.

Digital Library

[17]

Yolanda Vazquez-Alvarez, MatThew P Aylett, Stephen A Brewster, Rocio von Jungenfeld, and Antti Virolainen. 2014. Multilevel auditory displays for mobile eyes-free location-based interaction. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, 1567--1572.

Digital Library

[18]

Yolanda Vazquez-Alvarez and Stephen Brewster. 2009. Investigating Background & Foreground Interactions Using Spatial Audio Cues. In CHI'09 Extended Abstracts on Human Factors in Computing Systems (CHI EA'09). ACM.

Digital Library

[19]

Brian A Wandell. 1995. Foundations of Vision. Vol. 8. Sinauer Associates Sunderland.

[20]

P Zahorik, C Tam, K Wang, P Bangayan, and V Sundareswaran. 2001. Localization Accuracy in 3D Sound Displays: The Role of Visual-feedback Training. In Proceedings of The Advanced Displays and Interactive Displays Federal Laboratory Consortium.

Cited By

Nowak KTankelevitch LTang JRintel S(2023)Hear We Are: Spatial Audio Benefits Perceptions of Turn-Taking and Social Presence in Video MeetingsProceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3596671.3598578(1-10)Online publication date: 13-Jun-2023
https://dl.acm.org/doi/10.1145/3596671.3598578
Alinaghi ARemaggi LKim H(2023)Spatial Audio Reconstruction for VR Applications Using a Combined Method Based on SIRR and RSAO Approaches2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP)10.1109/MMSP59012.2023.10337683(1-6)Online publication date: 27-Sep-2023
https://doi.org/10.1109/MMSP59012.2023.10337683
Neumayr TAugstein MKubicek B(2022)Territoriality in Hybrid CollaborationProceedings of the ACM on Human-Computer Interaction10.1145/35552246:CSCW2(1-37)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555224
Show More Cited By

Index Terms

Hearing Is Believing: Synthesizing Spatial Audio from Everyday Objects to Users
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

ATSI: Augmented and Tangible Sonic Interaction
TEI '15: Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction

This paper presents ATSI, a system for sonic augmentation of physical objects with spatialized sounds and their control by gestural interaction. The implementation combines tracking of the users' hands and head with commodity hardware, and binaural ...
There Is More to Avatars Than Visuals: Investigating Combinations of Visual and Auditory User Representations for Remote Collaboration in Augmented Reality

Supporting remote collaboration through augmented reality facilitates the experience of co-presence by presenting the collaborator’s avatar in the user’s physical environment. While visual user representations are continuously researched and advanced, ...
Phantom Materialization: A Novel Method to Enhance Stereo Audio Reproduction on Headphones

Loudspeaker reproduction systems are subject to a compromise between spatial realism and cost. By simulating loudspeaker reproduction on headphones, the resulting spatial realism is limited accordingly, despite the virtually unlimited spatial imaging ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AH2019: Proceedings of the 10th Augmented Human International Conference 2019

March 2019

301 pages

ISBN:9781450365475

DOI:10.1145/3311823

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AH2019

AH2019: Augmented Human International Conference 2019

March 11 - 12, 2019

Reims, France

Acceptance Rates

AH2019 Paper Acceptance Rate 26 of 53 submissions, 49%;

Overall Acceptance Rate 121 of 306 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
402
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)5

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nowak KTankelevitch LTang JRintel S(2023)Hear We Are: Spatial Audio Benefits Perceptions of Turn-Taking and Social Presence in Video MeetingsProceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3596671.3598578(1-10)Online publication date: 13-Jun-2023
https://dl.acm.org/doi/10.1145/3596671.3598578
Alinaghi ARemaggi LKim H(2023)Spatial Audio Reconstruction for VR Applications Using a Combined Method Based on SIRR and RSAO Approaches2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP)10.1109/MMSP59012.2023.10337683(1-6)Online publication date: 27-Sep-2023
https://doi.org/10.1109/MMSP59012.2023.10337683
Neumayr TAugstein MKubicek B(2022)Territoriality in Hybrid CollaborationProceedings of the ACM on Human-Computer Interaction10.1145/35552246:CSCW2(1-37)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555224
Popoveniuc BVatavu R(2022)Transhumanism as a Philosophical and Cultural Framework for Extended Reality Applied to Human Augmentation13th Augmented Human International Conference10.1145/3532525.3532528(1-8)Online publication date: 26-May-2022
https://dl.acm.org/doi/10.1145/3532525.3532528
Buck LVargas MMcDonnell R(2022)The Effect of Spatial Audio on the Virtual Representation of Personal Space2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW55335.2022.00079(354-356)Online publication date: Mar-2022
https://doi.org/10.1109/VRW55335.2022.00079
Rajguru CBrianza GMemoli G(2022)Sound localization in web-based 3D environmentsScientific Reports10.1038/s41598-022-15931-y12:1Online publication date: 15-Jul-2022
https://doi.org/10.1038/s41598-022-15931-y
Bandukda MHolloway CTentori MWeibel NVan Laerhoven KAbowd GSalim F(2020)Audio AR to support nature connectedness in people with visual disabilitiesAdjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers10.1145/3410530.3414332(204-207)Online publication date: 10-Sep-2020
https://dl.acm.org/doi/10.1145/3410530.3414332
Watanabe HTerada TZhou BCheng JStrohmeier PIshimaru SKosch T(2020)Manipulatable Auditory Perception in Wearable ComputingProceedings of the Augmented Humans International Conference10.1145/3384657.3384792(1-12)Online publication date: 16-Mar-2020
https://dl.acm.org/doi/10.1145/3384657.3384792
Yang JSasikumar PBai HBarde ASörös GBillinghurst M(2020)The effects of spatial auditory and visual cues on mixed reality remote collaborationJournal on Multimodal User Interfaces10.1007/s12193-020-00331-1Online publication date: 28-Jun-2020
https://doi.org/10.1007/s12193-020-00331-1
Yang JMattern FHarle RFarrahi KLane N(2019)Audio augmented reality for human-object interactionsAdjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341162.3349302(408-412)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341162.3349302

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents