research-article

Open access

How We Swipe: A Large-scale Shape-writing Dataset and Empirical Findings

Authors:

Antti OulasvirtaAuthors Info & Claims

MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction

Article No.: 11, Pages 1 - 13

https://doi.org/10.1145/3447526.3472059

Published: 27 September 2021 Publication History

Abstract

Despite the prevalence of shape-writing (gesture typing, swype input, or swiping for short) as a text entry method, there are currently no public datasets available. We report a large-scale dataset that can support efforts in both empirical study of swiping as well as the development of better intelligent text entry techniques. The dataset was collected via a web-based custom virtual keyboard, involving 1,338 users who submitted 11,318 unique English words. We report aggregate-level indices on typing performance, user-related factors, as well as trajectory-level data, such as the gesture path drawn on top of the keyboard or the time lapsed between consecutively swiped keys. We find some well-known effects reported in previous studies, for example that speed and error are affected by age and language skill. We also find surprising relationships such that, on large screens, swipe trajectories are longer but people swipe faster.

References

[1]

J. Alvina, C. F. Griggio, X. Bi, and W. E. Mackay. 2017. CommandBoard: Creating a general-purpose command gesture input space for soft keyboard. In Proc. UIST. 17–28.

Digital Library

[2]

A. S. Arif and A. Mazalek. 2016. WebTEM: A Web Application to Record Text Entry Metrics. In Proc. ISS. 415–420.

[3]

S. Azenkot and S. Zhai. 2012. Touch Behavior with Different Postures on Soft Smartphone Keyboards. In Proc. MobileHCI. 251–260.

[4]

N. Banovic, V. Rao, A. Saravanan, A. K. Dey, and J. Mankoff. 2017. Quantifying Aversion to Costly Typing Errors in Expert Mobile Text Entry. In Proc. CHI. 4229–4241.

[5]

M. Beaudouin-Lafon, S. Huot, H. Olafsdottir, and P. Dragicevic. 2014. GlideCursor: Pointing with an Inertial Cursor. In Proc. AVI. 49–56.

[6]

J. Bergstrom-Lehtovirta and A. Oulasvirta. 2014. Modeling the functional area of the thumb on mobile touchscreen surfaces. In Proc. CHI. 1991–2000.

[7]

X. Bi, S. Azenkot, K. Partridge, and S. Zhai. 2013. Octopus: Evaluating Touchscreen Keyboard Correction and Recognition Algorithms Via “Remulation”. In Proc. CHI. 543–552.

[8]

M. Borenstein. 2009. Effect sizes for continuous data. In The handbook of research synthesis and meta-analysis (2nd ed.), H. Cooper, L. V. Hedges, and J. C. Valentine (Eds.). Sage Foundation, 221–235.

[9]

D. Buschek, B. Bisinger, and F. Alt. 2018. ResearchIME: A Mobile Keyboard Application for Studying Free Typing Behaviour in the Wild. In Proc. CHI. 1–14.

[10]

K. Caine. 2016. Local Standards for Sample Size at CHI. In Proc. CHI. 981–992.

Digital Library

[11]

W. Cui, J. Zheng, B. Lewis, D. Vogel, and X. Bi. 2019. HotStrokes: Word-gesture shortcuts on a trackpad. In Proc. CHI. 1–13.

[12]

[12] M. Davies. 2018. The corpus of contemporary American English: 1990-present. (2018).

[13]

V. Dhakal, A. M. Feit, P. O. Kristensson, and A. Oulasvirta. 2018. Observations on Typing from 136 Million Keystrokes. In Proc. CHI. 1–12.

[14]

M. D. Dunlop, E. Nicol, A. Komninos, P. Dona, and N. Durga. 2015. Measuring inviscid text entry using image description tasks. In Proc. CHI Workshop on Inviscid Text Entry and Beyond.

[15]

L. Findlater and L. Zhang. 2020. Input Accessibility: A Large Dataset and Summary Analysis of Age, Motor Ability and Input Performance. In Proc. SIGACCESS. 1–6.

[16]

A. Fowler, K. Partridge, C. Chelba, X. Bi, T. Ouyang, and S. Zhai. 2015. Effects of Language Modeling and Its Personalization on Touchscreen Typing Performance. In Proc. CHI. 649–658.

[17]

M. Franco-Salvador and L. A. Leiva. 2018. Multilingual Phrase Sampling for Text Entry Evaluations. Int. J. Hum.-Comput. Stud. 113, 1 (2018).

Digital Library

[18]

[18] D. R. Gentner, J. T. Grudin, S. Larochelle, D. A. Norman, and D. E. Rumelhart. 1983. Cognitive Aspects of Skilled Typewriting. Springer, Chapter A Glossary of Terms Including a Classification of Typing Errors, 39–43.

[19]

I. E. González, J. O. Wobbrock, D. H. Chau, A. Faulring, and B. A. Myers. 2007. Eyes on the Road, Hands on the Wheel: Thumb-based Interaction Techniques for Input on Steering Wheels. In Proc. GI. 95–102.

Digital Library

[20]

A. Gupta, C. Ji, H.-S. Yeo, A. Quigley, and D. Vogel. 2019. RotoSwype: Word-gesture typing using a ring. In Proc. CHI. 1–12.

[21]

N. Henze, E. Rukzio, and S. Boll. 2012. Observational and Experimental Investigation of Typing Behaviour Using Virtual Keyboards for Mobile Devices. In Proc. CHI. 2659–2668.

[22]

P. Isokoski and R. Raisamo. 2000. Device Independent Text Input: A Rationale and an Example. In Proc. AVI. 76–83.

[23]

C.-M. Karat, C. Halverson, D. Horn, and J. Karat. 1999. Patterns of Entry and Correction in Large Vocabulary Continuous Speech Recognition Systems. In Proc. CHI. 568–575.

[24]

P. O. Kristensson and K. Vertanen. 2012. Performance Comparisons of Phrase Sets and Presentation Styles for Text Entry Evaluations. In Proc. IUI. 29–32.

[25]

P. O. Kristensson and S. Zhai. 2004. SHARK2: A Large Vocabulary Shorthand Writing System for Pen-based Computers. In Proc. UIST. 43–52.

[26]

P. O. Kristensson and S. Zhai. 2007. Command strokes with and without preview: using pen gestures on keyboard for command selection. In Proc. CHI. 1137–1146.

[27]

L. A. Leiva, V. Romero, A. H. Toselli, and E. Vidal. 2011. Evaluating an Interactive-Predictive Paradigm on Handwriting Transcription: A Case Study and Lessons Learned. In Proc. COMPSAC. 610–617.

[28]

L. A. Leiva and G. Sanchis-Trilles. 2014. Representatively Memorable: Sampling the Right Phrase Set to Get the Text Entry Experiment Right. In Proc. CHI. 1709–1712.

[29]

I. S. MacKenzie, T. Kauppinen, and M. Silfverberg. 2001. Accuracy Measures for Evaluating Computer Pointing Devices. In Proc. CHI. 9–16.

[30]

I. S. MacKenzie and R. W. Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In Proc. CHI EA. 754–755.

Digital Library

[31]

A. Markussen, M. R. Jakobsen, and K. Hornbæk. 2014. Vulture: A Mid-air Word-gesture Keyboard. In Proc. CHI. 1073–1082.

[32]

[32] P. Norvig. 2019. Natural Language Corpus Data. O’Reilly Media, Inc., 219–242.

[33]

T. Paek and B.-J. P. Hsu. 2011. Sampling Representative Phrase Sets for Text Entry Experiments: A Procedure and Public Resource. In Proc. CHI. 2477–2480.

Digital Library

[34]

K. Palin, A. M. Feit, S. Kim, P. O. Kristensson, and A. Oulasvirta. 2019. How Do People Type on Mobile Devices? Observations from a Study with 37,000 Volunteers. In Proc. MobileHCI. 1–12.

[35]

P. Parhi, A. K. Karlson, and B. B. Bederson. 2006. Target Size Study for One-Handed Thumb Use on Small Touchscreen Devices. In Proc. MobileHCI. 203–210.

Digital Library

[36]

P. Quinn and S. Zhai. 2016. A Cost-Benefit Study of Text Entry Suggestion Interaction. In Proc. CHI. 83–88.

[37]

P. Quinn and S. Zhai. 2018. Modeling Gesture-Typing Movements. Hum.-Comput. Interact. 33, 3 (2018).

[38]

K. Reinecke and K. Z. Gajos. 2015. LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples. In Proc. CSCW. 1364–1378.

Digital Library

[39]

S. Reyal, S. Zhai, and P. O. Kristensson. 2015. Performance and User Experience of Touchscreen and Gesture Keyboards in a Lab Setting and in the Wild. In Proc. CHI. 679–688.

Digital Library

[40]

K. Tanaka-Ishii, D. Hayakawa, and M. Takeichi. 2003. Acquiring Vocabulary for Predictive Text Entry Through Dynamic Reuse of a Small User Corpus. In Proc. ACL. 407–414.

[41]

[41] P. D. Varcholik, J. J. LaViola, and C. E. Hughes. 2012. Establishing a baseline for text entry for a multi-touch virtual keyboard. Vol. 70. 657–672 pages.

[42]

M. Vasiljevas, J. Šalkevičius, T. Gedminas, and R. Damaševičius. 2015. A Prototype Gaze-Controlled Speller for Text Entry. In Proc. SYSTEM. 79–83.

[43]

K. Vertanen and P. O. Kristensson. 2011. A versatile dataset for text entry evaluations based on genuine mobile emails. In Proc. MobileHCI. 295–298.

Digital Library

[44]

K. Vertanen and P. O. Kristensson. 2014. Complementing Text Entry Evaluations with a Composition Task. ACM Trans. Comput.-Hum. Interact. 21, 2 (2014).

[45]

J. O. Wobbrock, A. D. Wilson, and Y. Li. 2007. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. In Proc. UIST. 159–168.

[46]

H.-S. Yeo, X.-S. Phang, S. J. Castellucci, P. O. Kristensson, and A. Quigley. 2017. Investigating tilt-based gesture keyboard entry for single-handed text entry on large devices. In Proc. CHI. 4194–4202.

[47]

F. Zamora-Martínez, V. Frinken, S. España Boquera, M. J. Castro-Bleda, A. Fischer, and H. Bunke. 2014. Neural Network Language Models for Off-Line Handwriting Recognition. Pattern Recogn. 47, 4 (2014).

[48]

S. Zhai and P. O. Kristensson. 2003. Shorthand Writing on Stylus Keyboard. In Proc. CHI. 97–104.

Digital Library

[49]

S. Zhai, A. Sue, and J. Accot. 2002. Movement Model, Hits Distribution and Learning in Virtual Keyboarding. In Proc. CHI. 17–24.

[50]

S. Zhu, J. Zheng, S. Zhai, and X. Bi. 2019. i’sFree: Eyes-Free Gesture Typing via a Touch-Enabled Remote Control. In Proc. CHI. 1–12.

Cited By

Richardson MBotros FShi YSnow BGuo PZhang LDong JVertanen KMa SWang R(2024)StegoType: Surface Typing from Egocentric CamerasAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686762(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686762
Richardson MBotros FShi YGuo PSnow BZhang LDong JVertanen KMa SWang R(2024)StegoType: Surface Typing from Egocentric CamerasProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676343(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676343
Yamanaka SUsuba HSato J(2024)Behavioral Differences between Tap and Swipe: Observations on Time, Error, Touch-point Distribution, and Trajectory for Tap-and-swipe Enabled TargetsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642272(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642272
Show More Cited By

Recommendations

B2B-Swipe: Swipe Gesture for Rectangular Smartwatches from a Bezel to a Bezel
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

We present B2B-Swipe, a single-finger swipe gesture for a rectangular smartwatch that starts at a bezel and ends at a bezel to enrich input vocabulary. There are 16 possible B2B-Swipes because a rectangular smartwatch has four bezels. Moreover, B2B-...
i'sFree: Eyes-Free Gesture Typing via a Touch-Enabled Remote Control
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Entering text without having to pay attention to the keyboard is compelling but challenging due to the lack of visual guidance. We propose i'sFree to enable eyes-free gesture typing on a distant display from a touch-enabled remote control. i'sFree does ...
Bezel swipe: conflict-free scrolling and multiple selection on mobile touch screen devices
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Zooming user interfaces are increasingly popular on mobile devices with touch screens. Swiping and pinching finger gestures anywhere on the screen manipulate the displayed portion of a page, and taps open objects within the page. This makes navigation ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction

September 2021

637 pages

ISBN:9781450383288

DOI:10.1145/3447526

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

DGIST
Academy of Finland

Conference

MobileHCI '21

Sponsor:

SIGCHI

MobileHCI '21: 23rd International Conference on Mobile Human-Computer Interaction

September 27 - October 1, 2021

Toulouse & Virtual, France

Acceptance Rates

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
2,838
Total Downloads

Downloads (Last 12 months)1,877
Downloads (Last 6 weeks)676

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Richardson MBotros FShi YSnow BGuo PZhang LDong JVertanen KMa SWang R(2024)StegoType: Surface Typing from Egocentric CamerasAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686762(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686762
Richardson MBotros FShi YGuo PSnow BZhang LDong JVertanen KMa SWang R(2024)StegoType: Surface Typing from Egocentric CamerasProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676343(1-14)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676343
Yamanaka SUsuba HSato J(2024)Behavioral Differences between Tap and Swipe: Observations on Time, Error, Touch-point Distribution, and Trajectory for Tap-and-swipe Enabled TargetsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642272(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642272
Shen JKhaldi KZhou ESurale HKarlson A(2024)Gesture2Text: A Generalizable Decoder for Word-Gesture Keyboards in XR Through Trajectory Coarse Discretization and Pre-TrainingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345619830:11(7118-7128)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456198
Shen JBoldu RKalla AGlueck MSurale HKarlson A(2024)RingGesture: A Ring-Based Mid-Air Gesture Typing System Powered by a Deep-Learning Word Prediction FrameworkIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345617930:11(7441-7451)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456179
Yfantidou SSpathis DConstantinides MXia Tvan Berkel N(2023)FairComp: Workshop on Fairness and Robustness in Machine Learning for Ubiquitous ComputingAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3605107(777-783)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3594739.3605107
Yfantidou SConstantinides MSpathis DVakali AQuercia DKawsar F(2023)The State of Algorithmic Fairness in Mobile Human-Computer InteractionProceedings of the 25th International Conference on Mobile Human-Computer Interaction10.1145/3565066.3608685(1-7)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1145/3565066.3608685
Chu JAn DMa YCui WZhai SGu XBi X(2023)WordGesture-GAN: Modeling Word-Gesture Movement with Generative Adversarial NetworkProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581279(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581279
Dudley JZheng JGupta ABenko HLongest MWang RKristensson P(2023)Evaluating the Performance of Hand-Based Probabilistic Text Input Methods on a Mid-Air Virtual Qwerty KeyboardIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332023829:11(4567-4577)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3320238
Shen JDudley JKristensson P(2023)Fast and Robust Mid-Air Gesture Typing for AR Headsets using 3D Trajectory DecodingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332021829:11(4622-4632)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3320218
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents