research-article

Tap, Dwell or Gesture?: Exploring Head-Based Text Entry Techniques for HMDs

Authors:

Yuanchun ShiAuthors Info & Claims

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Pages 4479 - 4488

https://doi.org/10.1145/3025453.3025964

Published: 02 May 2017 Publication History

Abstract

Despite the increasing popularity of head mounted displays (HMDs), development of efficient text entry methods on these devices has remained under explored. In this paper, we investigate the feasibility of head-based text entry for HMDs, by which, the user controls a pointer on a virtual keyboard using head rotation. Specifically, we investigate three techniques: TapType, DwellType, and GestureType. Users of TapType select a letter by pointing to it and tapping a button. Users of DwellType select a letter by pointing to it and dwelling over it for a period of time. Users of GestureType perform word-level input using a gesture typing style. Two lab studies were conducted. In the first study, users typed 10.59 WPM, 15.58 WPM, and 19.04 WPM with DwellType, TapType, and GestureType, respectively. Users subjectively felt that all three of the techniques were easy to learn and considered the induced fatigue to be acceptable. In the second study, we further investigated GestureType. We improved its gesture-word recognition algorithm by incorporating the head movement pattern obtained from the first study. This resulted in users reaching 24.73 WPM after 60 minutes of training. Based on these results, we argue that head-based text entry is feasible and practical on HMDs, and deserves more attention.

Supplementary Material

suppl.mov (pn4116p.mp4)

Supplemental video

Download
2.49 MB

MP4 File (p4479-yu.mp4)

Download
188.27 MB

References

[1]

2011. The Open American National Corpus. http://www.anc.org/

[2]

Ouais Alsharif, Tom Ouyang, Francçoise Beaufays, Shumin Zhai, Thomas Breuel, Johan Schalkwyk. Long short term memory neural network for keyboard gesture decoding. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015. 2076--2080

[3]

Christoph Amma, Marcus Georgi, and Tanja Schultz. 2012. Hands-Free Mobile Text Input by Spotting and Continuous Recognition of 3d-Space Handwriting with Inertial Sensors. In 2012 16th International Symposium on Wearable Computers (ISWC'12), 52--59.

Digital Library

[4]

Xiaojun Bi, Ciprian Chelba, Tom Ouyang, Kurt Partridge, and Shumin Zhai. 2012. Bimanual gesture keyboard. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST '12). 137--146.

Digital Library

[5]

Doug A. Bowman, Vinh Q. Ly, and Joshua M. Campbell. 2001. Pinch keyboard: Natural text input for immersive virtual environments.

[6]

Xiang 'Anthony' Chen, Tovi Grossman, and George Fitzmaurice. 2014. Swipeboard: a text entry technique for ultra-small interfaces that supports novice to expert transitions. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). 615--620.

Digital Library

[7]

Weiya Chen, Anthony Plancoulaine, Nicolas Férey, Damien Touraine, Julien Nelson, and Patrick Bourdot. 2013. 6DoF navigation in virtual worlds: comparison of joystick-based and head-controlled paradigms. In Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology(VRST '13). 111--114.

Digital Library

[8]

Wenxin Feng, Ming Chen, and Margrit Betke. 2014. Target reverse crossing: a selection method for camerabased mouse-replacement systems. In Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '14). Article 39, 4 pages.

Digital Library

[9]

Yulia Gizatdinova, Oleg Špakov, and Veikko Surakka. 2012. Comparison of video-based pointing and selection techniques for hands-free text entry. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '12), Genny Tortora, Stefano Levialdi, and Maurizio Tucci (Eds.). 132--139.

Digital Library

[10]

Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language modeling for soft keyboards. In Proceedings of the 7th international conference on Intelligent user interfaces (IUI '02). 194195.

Digital Library

[11]

Mitchell Gordon, Tom Ouyang, and Shumin Zhai. 2016. WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). 3817--3821.

Digital Library

[12]

Tovi Grossman, Xiang Anthony Chen, and George Fitzmaurice. 2015. Typing on Glasses: Adapting Text Entry to Smart Eyewear. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). 144--152.

Digital Library

[13]

John Paulin Hansen, Kristian Tørning, Anders Sewerin Johansen, Kenji Itoh, and Hirotaka Aoki. 2004. Gaze typing compared with input by head and hand. In Proceedings of the 2004 symposium on Eye tracking research & applications (ETRA '04). 131--138.

Digital Library

[14]

Per Ola Kristensson and Keith Vertanen. 2012. The potential of dwell-free eye-typing for fast assistive gaze communication. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '12), Stephen N. Spencer (Ed.). 241--244.

Digital Library

[15]

Per Ola Kristensson and Shumin Zhai. 2004. SHARK2 : a large vocabulary shorthand writing system for penbased computers. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST '04). 43--52.

Digital Library

[16]

Andrew Kurauchi, Wenxin Feng, Ajjen Joshi, Carlos Morimoto, and Margrit Betke. 2016. EyeSwipe: Dwellfree Text Entry Using Gaze Paths. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). 1952--1956.

Digital Library

[17]

I. Scott MacKenzie and R. William Soukoreff. 2002. A character-level error analysis technique for evaluating text entry methods. In Proceedings of the second Nordic conference on Human-computer interaction (NordiCHI '02). 243--246.

Digital Library

[18]

I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). 754--755.

Digital Library

[19]

Päivi Majaranta and Kari-Jouko Räihä. 2007. Text entry by gaze: Utilizing eye-tracking. Text entry systems: Mobility, accessibility, universality: 175--187.

[20]

Päivi Majaranta, Ulla-Kaija Ahola, and Oleg Špakov. 2009. Fast gaze typing with an adjustable dwell time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). 357--360.

Digital Library

[21]

Anders Markussen, Mikkel R. Jakobsen, and Kasper Hornbæk. 2013. Selection-based mid-air text entry on large displays. In IFIP Conference on HumanComputer Interaction. 401--418.

[22]

Anders Markussen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2014. Vulture: a mid-air wordgesture keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). 1073--1082.

Digital Library

[23]

Roderick McCall, Benoît Martin, Andrei Popleteev, Nicolas Louveton and Thomas Engel. 2015. Text entry on smart glasses. In 2015 8th International Conference on Human System Interaction (HSI). 195--200.

[24]

Tao Ni, Doug Bowman, and Chris North. 2011. AirStroke: bringing unistroke text entry to freehand gesture interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). 2473--2476.

Digital Library

[25]

Paul Nation and Robert Waring. 1997. Vocabulary size, text coverage and word lists. Vocabulary: Description, acquisition and pedagogy Rev 14: 6--19.

[26]

Diogo Pedrosa, Maria Da Graça Pimentel, Amy Wright, and Khai N. Truong. 2015. Filteryedping: Design Challenges and User Performance of DwellFree Eye Typing. ACM Trans. Access. Comput. 6, 1, Article 3 (March 2015), 37 pages.

Digital Library

[27]

Shyam Reyal, Shumin Zhai, and Per Ola Kristensson. 2015. Performance and User Experience of Touchscreen and Gesture Keyboards in a Lab Setting and in the Wild. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). 679--688.

Digital Library

[28]

Sayan Sarcar, Prateek Panwar, and Tuhin Chakraborty. 2013. EyeK: an efficient dwell-free eye gaze-based text entry system. In Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction (APCHI '13). 215--220.

Digital Library

[29]

Alexander Schick, Daniel Morlock, Christoph Amma, Tanja Schultz, and Rainer Stiefelhagen. 2012. Visionbased handwriting recognition for unrestricted text input in mid-air. In Proceedings of the 14th ACM international conference on Multimodal interaction (ICMI '12). 217--220.

Digital Library

[30]

Srinath Sridhar, Anna Maria Feit, Christian Theobalt, and Antti Oulasvirta. 2015. Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). 3643--3652.

Digital Library

[31]

Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: Investigating Fast Mobile Text Entry using Sentence-Based Decoding of Touchscreen Keyboard Input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). 659--668.

Digital Library

[32]

Daniel Vogel and Patrick Baudisch. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). 657666.

Digital Library

[33]

Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). 539--548.

Digital Library

[34]

Chun Yu, Ke Sun, Mingyuan Zhong, Xincheng Li, Peijun Zhao, and Yuanchun Shi. 2016. OneDimensional Handwriting: Inputting Letters and Words on Smart Glasses. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). 71--82.

Digital Library

[35]

Shumin Zhai, Per Ola Kristensson, Pengjun Gong, Michael Greiner, Shilei Allen Peng, Liang Mico Liu, and Anthony Dunnigan. 2009. Shapewriter on the iphone: from the laboratory to the real world. InCHI '09 Extended Abstracts on Human Factors in Computing Systems (CHI EA '09). 2667--2670.

Digital Library

[36]

Shumin Zhai, Carlos Morimoto, and Steven Ihde. 1999. Manual and gaze input cascaded (MAGIC) pointing. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '99). 246--253.

Digital Library

[37]

Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand writing on stylus keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). 97--104.

Digital Library

[38]

Shumin Zhai and Per Ola Kristensson. 2012. The word-gesture keyboard: reimagining keyboard interaction. Commun. ACM 55, 9 (September 2012): 91--101.

Digital Library

Cited By

Zhang JChen TGong WLiu JChen J(2024)Exploring Data Input Problems in Mixed Reality Environments: Proposal and Evaluation of Natural Interaction TechniquesFuture Internet10.3390/fi1605015016:5(150)Online publication date: 27-Apr-2024
https://doi.org/10.3390/fi16050150
Rasanji VKumarasinghe NPethangoda RKarunarathna VRodrigo R(2024)Virtual Reality Bank: A Novel Banking Experience in Immersive WorldProceedings of the 26th Symposium on Virtual and Augmented Reality10.1145/3691573.3691576(193-202)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3691573.3691576
Akhoroz MYildirim C(2024)Poke Typing: Effects of Hand-Tracking Input and Key Representation on Mid-Air Text Entry Performance in Virtual RealityProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685734(293-301)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685734
Show More Cited By

Index Terms

Tap, Dwell or Gesture?: Exploring Head-Based Text Entry Techniques for HMDs
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

The popularity of mobile devices with large screens is making single-handed interaction difficult. We propose and evaluate a novel design point around a tilt-based text entry technique which supports single handed usage. Our technique is based on the ...
Investigating Gesture Typing for Indirect Touch

With the development of ubiquitous computing, entering text on HMDs and smart TVs using handheld touchscreen devices (e.g., smartphone and controller) is becoming more and more attractive. In these indirect touch scenarios, the touch input surface is ...
DRG-Keyboard: Enabling Subtle Gesture Typing on the Fingertip with Dual IMU Rings

We present DRG-Keyboard, a gesture keyboard enabled by dual IMU rings, allowing the user to swipe the thumb on the index fingertip to perform word gesture typing as if typing on a miniature QWERTY keyboard. With dual IMUs attached to the user's thumb and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

May 2017

7138 pages

ISBN:9781450346559

DOI:10.1145/3025453

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Plan
Tsinghua University Research Funding
Natural Science Foundation of China

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

124
Total Citations
View Citations
1,858
Total Downloads

Downloads (Last 12 months)239
Downloads (Last 6 weeks)20

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang JChen TGong WLiu JChen J(2024)Exploring Data Input Problems in Mixed Reality Environments: Proposal and Evaluation of Natural Interaction TechniquesFuture Internet10.3390/fi1605015016:5(150)Online publication date: 27-Apr-2024
https://doi.org/10.3390/fi16050150
Rasanji VKumarasinghe NPethangoda RKarunarathna VRodrigo R(2024)Virtual Reality Bank: A Novel Banking Experience in Immersive WorldProceedings of the 26th Symposium on Virtual and Augmented Reality10.1145/3691573.3691576(193-202)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3691573.3691576
Akhoroz MYildirim C(2024)Poke Typing: Effects of Hand-Tracking Input and Key Representation on Mid-Air Text Entry Performance in Virtual RealityProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685734(293-301)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685734
Darbar RHu XYan XWei YLiang HXu WSarcar S(2024)OnArmQWERTY: An Empirical Evaluation of On-Arm Tap Typing for AR HMDsProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682084(1-12)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682084
Zhang DLi ZAshok VSeiple WRamakrishnan IBi X(2024)Accessible Gesture Typing on Smartphones for People with Low VisionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676447(1-11)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676447
Lertvittayakumjorn PCai SDou BHo CZhai S(2024)Can Capacitive Touch Images Enhance Mobile Keyboard Decoding?Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676420(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676420
Wu LLafreniere BGrossman TWhite TSantosa S(2024)Body Language for VUIs: Exploring Gestures to Enhance Interactions with Voice User InterfacesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660691(133-150)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660691
Wan TZhang LYang HIrani PYu LLiang H(2024)Exploration of Foot-based Text Entry Techniques for Virtual Reality EnvironmentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642757(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642757
Chen LCai YWang RDing STang YHansen PSun L(2024)Supporting Text Entry in Virtual Reality with Large Language Models2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00073(524-534)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00073
Hu JDudley JKristensson P(2024)SkiMR: Dwell-free Eye Typing in Mixed Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00065(439-449)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00065
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents