research-article

Haptic and Visual feedback in 3D Audio Mixing Interfaces

Authors:

Steven Gelineck,

Dan OverholtAuthors Info & Claims

AM '15: Proceedings of the Audio Mostly 2015 on Interaction With Sound

Article No.: 14, Pages 1 - 6

https://doi.org/10.1145/2814895.2814918

Published: 07 October 2015 Publication History

Abstract

This paper describes the implementation and informal evaluation of a user interface that explores haptic feedback for 3D audio mixing. The implementation compares different approaches using either the LEAP Motion for mid-air hand gesture control, or the Novint Falcon for active haptic feedback in order to augment the perception of the 3D space. We compare different interaction paradigms implemented using these interfaces, aiming to increase speed and accuracy and reduce the need for constant visual feedback. While the LEAP Motion relies upon visual perception and proprioception, users can forego visual feedback with interfaces such as the Novint Falcon and rely primarily on haptic cues, allowing more focus on the spatial sound elements. Results of the evaluation support this claim, as users preferred the interaction paradigm using the Falcon with no visual feedback. Furthermore, users disliked active haptic feedback for augmented perception of 3D space or for snapping to objects.

References

[1]

E Berdahl, A Kontogeorgakopoulos, and D Overholt. Hsp v2: Haptic signal processing with extensions for physical modeling. In 5th International Workshop on Haptic and Audio Interaction Design-HAID, Copenhagen, pages 61--62, 2010.

[2]

Edgar Berdahl and Alexandros Kontogeorgakopoulos. The firefader design: Simple, open-source, and reconfigurable haptics for musicians. In Proceedings of the Sound and Music Computing Conference, pages 90--98, 2012.

[3]

Edgar Berdahl, Günter Niemeyer, and Julius O Smith. Using haptics to assist performers in making gestures to a musical instrument. In Proceedings of New Interfaces for Musical Expression, 2009.

[4]

Nicolas d'Alessandro, Joëlle Tilmanne, Ambroise Moreau, and Antonin Puleo. Airpiano: A multi-touch keyboard with hovering control. In Proceedings of New Interfaces for Musical Expression, 2015.

[5]

Steven Gelineck and Dannie Korsgaard. An exploratory evaluation of user interfaces for 3d audio mixing. In Audio Engineering Society Convention 138. Audio Engineering Society, 2015.

[6]

Steven Gelineck, Dan Overholt, Morten Büchert, and Jesper Andersen. Towards an interface for music mixing based on smart tangibles and multitouch. In Proceedings of New Interfaces for Musical Expression, 2013.

[7]

Jürgen Herre, Johannes Hilpert, Achim Kuntz, and Jan Plogsties. Mpeg-h audio---the new standard for universal spatial/3d audio coding. Journal of the Audio Engineering Society, 62(12):821--830, 2015.

[8]

Michal Lech and Bozena Kostek. Testing a novel gesture-based mixing interface. Journal of the Audio Engineering Society, 61(5):301--313, 2013.

[9]

Mark T Marshall, Joseph Malloch, and Marcelo M Wanderley. Non-conscious control of sound spatialization. In Proc. of the International Conference on Enactive Interfaces, 2007.

[10]

Justin Mathew, Stéphane Huot, and Alan Blum. A morphological analysis of audio objects and their control methods for 3d audio. In Proceedings of New Interfaces for Musical Expression, 2014.

[11]

Frank Melchior, Chris Pike, Matthew Brooks, and Stuart Grace. On the use of a haptic feedback device for sound source control in spatial audio systems. In Audio Engineering Society Convention 134. Audio Engineering Society, 2013.

[12]

Frank Melchior and Sascha Spors. Spatial audio reproduction: from theory to production. In Audio Engineering Convention 129, 2010.

[13]

William Moss and Bryan Cunitz. Haptic theremin: Developing a haptic musical controller using the sensable phantom omni. In Proceedings of the International Computer Music Conference, pages 275--277, 2005.

[14]

Jörg Müller, Matthias Geier, Christina Dicke, and Sascha Spors. The boomroom: mid-air direct interaction with virtual sound sources. In Proc. of the ACM conference on Human factors in Computing Systems, 2014.

Digital Library

[15]

Nils Peters, Georgios Marentakis, and Stephen McAdams. Current technologies and compositional practices for spatialization: A qualitative and quantitative analysis. Computer Music Journal, 35(1):10--27, 2011.

Digital Library

[16]

Jarrod Ratcliffe. Hand motion-controlled audio mixing interface. In Proceedings of New Interfaces for Musical Expression, 2014.

[17]

Stephen Sinclair. Force-feedback hand controllers for musical interaction. PhD thesis, McGill University, 2007.

[18]

Robert Tubb and Simon Dixon. An evaluation of multidimensional controllers for sound design tasks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 47--56. ACM, 2015.

Digital Library

[19]

Frank Weichert, Daniel Bachmann, Bartholomäus Rudak, and Denis Fisseler. Analysis of the accuracy and robustness of the leap motion controller. Sensors, 13(5):6380--6393, 2013.

Cited By

Husa PLeischner VKaltenbrunner MTomás E(2024)Interactive Spatial Audio Diffusion: The Pendulum Haptic Interface PrototypeDigital Interaction and Machine Intelligence10.1007/978-3-031-66594-3_17(164-171)Online publication date: 15-Aug-2024
https://doi.org/10.1007/978-3-031-66594-3_17
Jiang SLim LSra M(2023)Spatializing Music in Virtual RealityProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3618011(1-3)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3618011
Turchet LHamilton RCamci A(2021)Music in Extended RealitiesIEEE Access10.1109/ACCESS.2021.30529319(15810-15832)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3052931
Show More Cited By

Index Terms

Haptic and Visual feedback in 3D Audio Mixing Interfaces
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Haptic Fidelity Framework: Defining the Factors of Realistic Haptic Feedback for Virtual Reality
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Providing haptic feedback in virtual reality to make the experience more realistic has become a strong focus of research in recent years. The resulting haptic feedback systems differ greatly in their technologies, feedback possibilities, and overall ...
Evaluation of haptic virtual reality user interfaces for medical marking on 3D models
Highlights
- Two haptic VR interfaces are proposed to interact with volumetric medical images for improving current computer-aided medical diagnosis and planning.
- Compared with the traditional 2D interface using a screen-based 2D display and a ...
Abstract
Three-dimensional (3D) visualization has been widely used in computer-aided medical diagnosis and planning. To interact with 3D models, current user interfaces in medical systems mainly rely on the traditional 2D interaction techniques by ...
Haptic feedback with HaptoBend: utilizing shape-change to enhance virtual reality
SUI '17: Proceedings of the 5th Symposium on Spatial User Interaction

HaptoBend¹ is an original device for providing general passive haptic feedback (PHF) in virtual reality (VR). The importance of haptic feedback in VR is apparent through improvements in presence [3] and user performance [2]. However, most general haptic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AM '15: Proceedings of the Audio Mostly 2015 on Interaction With Sound

October 2015

250 pages

ISBN:9781450338967

DOI:10.1145/2814895

Conference Chair:
George Kalliris
Aristotle University of Thessaloniki
,
Program Chair:
Charalampos Dimoulas
Aristotle University of Thessaloniki

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AM '15

AM '15: Audio Mostly 2015

October 7 - 9, 2015

Thessaloniki, Greece

Acceptance Rates

Overall Acceptance Rate 177 of 275 submissions, 64%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
309
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)4

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Husa PLeischner VKaltenbrunner MTomás E(2024)Interactive Spatial Audio Diffusion: The Pendulum Haptic Interface PrototypeDigital Interaction and Machine Intelligence10.1007/978-3-031-66594-3_17(164-171)Online publication date: 15-Aug-2024
https://doi.org/10.1007/978-3-031-66594-3_17
Jiang SLim LSra M(2023)Spatializing Music in Virtual RealityProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3618011(1-3)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3618011
Turchet LHamilton RCamci A(2021)Music in Extended RealitiesIEEE Access10.1109/ACCESS.2021.30529319(15810-15832)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3052931
Moffat D(2021)AI Music Mixing SystemsHandbook of Artificial Intelligence for Music10.1007/978-3-030-72116-9_13(345-375)Online publication date: 3-Jul-2021
https://doi.org/10.1007/978-3-030-72116-9_13
Haas GStemasov ERietzler MRukzio EWakkary RAndersen KOdom WDesjardins APetersen M(2020)Interactive Auditory Mediated RealityProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395493(2035-2050)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395493
Mathew JHuot SKatz B(2017)Survey and implications for the design of new 3D audio production and authoring toolsJournal on Multimodal User Interfaces10.1007/s12193-017-0245-z11:3(277-287)Online publication date: 29-Apr-2017
https://doi.org/10.1007/s12193-017-0245-z

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents