Article

Multimodal 'eyes-free' interaction techniques for wearable devices

Authors:

Stephen Brewster,

Joanna Lumsden,

Stuart TaskerAuthors Info & Claims

CHI '03: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 473 - 480

https://doi.org/10.1145/642611.642694

Published: 05 April 2003 Publication History

Abstract

Mobile and wearable computers present input/output prob-lems due to limited screen space and interaction techniques. When mobile, users typically focus their visual attention on navigating their environment - making visually demanding interface designs hard to operate. This paper presents two multimodal interaction techniques designed to overcome these problems and allow truly mobile, 'eyes-free' device use. The first is a 3D audio radial pie menu that uses head gestures for selecting items. An evaluation of a range of different audio designs showed that egocentric sounds re-duced task completion time, perceived annoyance, and al-lowed users to walk closer to their preferred walking speed. The second is a sonically enhanced 2D gesture recognition system for use on a belt-mounted PDA. An evaluation of the system with and without audio feedback showed users' ges-tures were more accurate when dynamically guided by au-dio-feedback. These novel interaction techniques demon-strate effective alternatives to visual-centric interface de-signs on mobile devices.

References

[1]

Barfield, W. and Caudell, T. (eds.). Fundamentals of wearable computers and augmented reality. Lawrence Erlbaum Associates, Mahwah, New Jersey, 2001.]]

Digital Library

[2]

Begault, D.R. 3-D sound for virtual reality and multime-dia. Academic Press, Cambridge, MA, 1994.]]

Digital Library

[3]

Brewster, S.A. Overcoming the Lack of Screen Space on Mobile Computers. Personal and Ubiquitous Computing, 6 (3). 188--205.]]

Digital Library

[4]

Brewster, S.A. Using Non-Speech Sounds to Provide Navigation Cues. ACM Transactions on Computer-Human Interaction, 5 (3). 224--259.]]

Digital Library

[5]

Cohen, M. and Ludwig, L.F. Multidimensional audio window management. International Journal of Man-Machine Studies, 34. 319--336.]]

Digital Library

[6]

Fiedlander, N., Schlueter, K. and Mantei, M., Bullseye! When Fitt's law doesn't fit. in Proceedings of ACM CHI'98, (Los Angeles, CA, 1998), ACM Press Addison-Wesley, 257--264.]]

Digital Library

[7]

Geelhoed, E., Falahee, M. and Latham, K. Safety and comfort of eyeglass displays. in Thomas, P. and Gel-lersen, H.W. eds. Handheld and Ubiquitous Computing, Springer, Berlin, 2000, 236--247.]]

Digital Library

[8]

Harrison, B.L., Fishkin, K.P., Gujar, A., Mochon, C. and Want, R., Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces. in Proceedings of ACM CHI'98, (Los Angeles, CA, 1998), ACM Press Addison-Wesley, 17--24.]]

Digital Library

[9]

Hart, S.G. and Wickens, C. Workload assessment and prediction. in Booher, H.R. ed. MANPRINT, an ap-proach to systems integration, Van Nostrand Reinhold, New York, 1990, 257--296.]]

[10]

Knight, J. and Baber, C., Physical load and wearable computers. in Proceedings of the 5th International Sym-posium Wearable Computers, (Atlanta, GA, 2000), IEEE Computer Society.]]

[11]

Malkewitz, R., Head pointing and speech control as a hands-free interface to desktop computing. in Proceed-ings of ACM ASSETS 98, (Marina del Rey, CA, 1998), ACM Press, 182--188.]]

Digital Library

[12]

Petrie, H., Furner, S. and Strothotte, T., Design Lifecy-cles and Wearable Computers for Users with Disabili-ties. in First workshop on human-computer interaction with mobile devices, (Glasgow, UK, 1998), Glasgow University.]]

[13]

Pirhonen, A., Brewster, S.A. and Holguin, C., Gestural and Audio Metaphors as a Means of Control for Mobile Devices. in Proceedings of ACM CHI 2002, (Minneapo-lis, MN, 2002), ACM Press, 291--298.]]

Digital Library

[14]

Savidis, A., Stephanidis, C., Korte, A., Crispien, K. and Fellbaum, C., A generic direct-manipulation 3D-auditory environment for hierarchical navigation in non-visual in-teraction. in Proceedings of ACM ASSETS'96, (Van-couver, Canada, 1996), ACM Press, 117--123.]]

Digital Library

[15]

Sawhney, N. and Schmandt, C. Nomadic Radio: speech and audio interaction for contextual messaging in no-madic environments. ACM Transactions on Human-Computer Interaction, 7 (3). 353--383.]]

Digital Library

[16]

Willey, M. Gesture recognition algorithm, ETLA Tech-nical Services, Palo Alto, 2002.]]

Cited By

Houghton RMartinetti AMajumdar A(2024)A Framework for Selecting and Assessing Wearable Sensors Deployed in Safety Critical ScenariosSensors10.3390/s2414458924:14(4589)Online publication date: 15-Jul-2024
https://doi.org/10.3390/s24144589
Marentakis GBalic A(2024)Situational impairment due to walking with conversational versus graphical interfacesProceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures10.1145/3678299.3678307(77-85)Online publication date: 18-Sep-2024
https://dl.acm.org/doi/10.1145/3678299.3678307
van Rheden VPöckl LMeschtscherjakov AKostakos VKay JHoang T(2024)GestureShirt: Eyes-Free Interaction while RunningCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677557(244-248)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677557
Show More Cited By

Index Terms

Multimodal 'eyes-free' interaction techniques for wearable devices

Recommendations

Simulating an extendable tangible slider for eyes-free one-handed interaction on mobile devices
AVI '18: Proceedings of the 2018 International Conference on Advanced Visual Interfaces

Sliders are widely used on mobile devices. Envisioning mobile devices that can dynamically deform to raise tangible controls from the screen surface, tangible sliders offer the benefit of eyes-free interaction. However, reaching for distant values with ...
Deformable tangible slider for eyes-free one-handed thumb interaction on mobile devices
IHM '17: Proceedings of the 29th Conference on l'Interaction Homme-Machine

Graphical sliders are widely used on mobile devices. However, with a single hand, reaching for far values is difficult : users change their grip and can drop the device. Moreover, sliders require visual attention to operate them. Envisioning mobile ...
Eyes-free interaction with free-hand gestures and auditory menus

Auditory interfaces can overcome visual interfaces when a primary task, such as driving, competes for the attention of a user controlling a device, such as radio. In emerging interfaces enabled by camera tracking, auditory displays may also provide ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '03: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2003

620 pages

ISBN:1581136307

DOI:10.1145/642611

Conference Chairs:
Gilbert Cockton
University of Sunderland, UK
,
Panu Korhonen
Nokia Research Center, Finland

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 April 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI03

Sponsor:

CHI03: Human Factors in Computing Systems

April 5 - 10, 2003

Florida, Ft. Lauderdale, USA

Acceptance Rates

CHI '03 Paper Acceptance Rate 75 of 468 submissions, 16%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

181
Total Citations
View Citations
4,130
Total Downloads

Downloads (Last 12 months)87
Downloads (Last 6 weeks)11

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Houghton RMartinetti AMajumdar A(2024)A Framework for Selecting and Assessing Wearable Sensors Deployed in Safety Critical ScenariosSensors10.3390/s2414458924:14(4589)Online publication date: 15-Jul-2024
https://doi.org/10.3390/s24144589
Marentakis GBalic A(2024)Situational impairment due to walking with conversational versus graphical interfacesProceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures10.1145/3678299.3678307(77-85)Online publication date: 18-Sep-2024
https://dl.acm.org/doi/10.1145/3678299.3678307
van Rheden VPöckl LMeschtscherjakov AKostakos VKay JHoang T(2024)GestureShirt: Eyes-Free Interaction while RunningCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3677557(244-248)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3677557
Rey BSakamoto YSin JIrani P(2024)Understanding User Preferences of Voice Assistant Answer Structures for Personal Health Data QueriesProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665552(1-15)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665552
van Rheden VPöckl LMeschtscherjakov A(2024)GestureShirt: Exploring Gestures in Front of the Body for Truly Mobile Interaction while RunningAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680443(1-6)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680443
Cheng WChan L(2024)BodyTouchProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314267:4(1-22)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631426
Yang JShi YZhang YLi KRosli DJain AZhang SLi TLanday JLam M(2024)ReactGenie: A Development Framework for Complex Multimodal Interactions Using Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642517(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642517
Rey BLee BChoe EIrani P(2024)Databiting: Lightweight, Transient, and Insight Rich Exploration of Personal DataIEEE Computer Graphics and Applications10.1109/MCG.2024.335388844:2(65-72)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1109/MCG.2024.3353888
Zhang HYin LZhang H(2024)A real-time camera-based gaze-tracking system involving dual interactive modes and its application in gamingMultimedia Systems10.1007/s00530-023-01204-930:1Online publication date: 16-Jan-2024
https://dl.acm.org/doi/10.1007/s00530-023-01204-9
Chheda-Kothary ARios DSmith KReyna AZhang CSmith B(2023)Understanding Blind and Low Vision Users' Attitudes Towards Spatial Interactions in Desktop Screen ReadersProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3614490(1-5)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3614490
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents