research-article

FingerSound: Recognizing unistroke thumb gestures using a ring

Authors:

Anandghan Waghmare,

Scott Gilliland,

Thad E. Starner,

Gregory D. AbowdAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 1, Issue 3

Article No.: 120, Pages 1 - 19

https://doi.org/10.1145/3130985

Published: 11 September 2017 Publication History

Abstract

We introduce FingerSound, an input technology to recognize unistroke thumb gestures, which are easy to learn and can be performed through eyes-free interaction. The gestures are performed using a thumb-mounted ring comprising a contact microphone and a gyroscope sensor. A K-Nearest-Neighbor(KNN) model with a distance function of Dynamic Time Warping (DTW) is built to recognize up to 42 common unistroke gestures. A user study, where the real-time classification results were given, shows an accuracy of 92%-98% by a machine learning model built with only 3 training samples per gesture. Based on the user study results, we further discuss the opportunities, challenges and practical limitations of FingerSound when deploying it to real-world applications in the future.

Supplementary Material

zhang (zhang.zip)

Supplemental movie, appendix, image and software files for, FingerSound: Recognizing unistroke thumb gestures using a ring

Download
38.52 MB

References

[1]

Liwei Chan, Yi-Ling Chen, Chi-Hao Hsieh, Rong-Hao Liang, and Bing-Yu Chen. 2015. CyclopsRing: Enabling Whole-Hand and Context-Aware Interactions Through a Fisheye Ring. In Proceedings of the 28th Annual ACM Symposium on User Interface Software 8 Technology (UIST '15). ACM, New York, NY, USA, 549--556.

Digital Library

[2]

Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3D Input Using Two Magnetic Sensors. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 237--244.

Digital Library

[3]

Gabe Cohn, Sidhant Gupta, Tien-Jui Lee, Dan Morris, Joshua R Smith, Matthew S Reynolds, Desney S Tan, and Shwetak N Patel. 2012. An ultra-low-power human body motion sensor using static electric field sensing. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 99--102.

Digital Library

[4]

Artem Dementyev and Joseph A. Paradiso. 2014. WristFlex: Low-power Gesture Input with Wrist-worn Pressure Sensors. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST ‘14. Association for Computing Machinery (ACM).

Digital Library

[5]

Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-handed Continuous Smartwatch Input Using Wrist Gestures. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 861--872.

Digital Library

[6]

Chris Harrison, Desney Tan, and Dan Morris. 2010. Skinput: appropriating the body as an input surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 453--462.

Digital Library

[7]

Da-Yuan Huang, Liwei Chan, Shuo Yang, Fan Wang, Rong-Hao Liang, De-Nian Yang, Yi-Ping Hung, and Bing-Yu Chen. 2016. DigitSpace: Designing Thumb-to-Fingers Touch Interfaces for One-Handed and Eyes-Free Interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI'16). ACM, New York, NY, USA, 1526--1537.

Digital Library

[8]

Howell Istance, Richard Bates, Aulikki Hyrskykari, and Stephen Vickers. 2008. Snap Clutch, a Moded Approach to Solving the Midas Touch Problem. In Proceedings of the 2008 Symposium on Eye Tracking Research 8 Applications (ETRA '08). ACM, New York, NY, USA, 221--228.

Digital Library

[9]

Aftab Khan, Nils Hammerla, Sebastian Mellor, and Thomas Plötz. 2016. Optimising sampling rates for accelerometer-based human activity recognition. Pattern Recognition Letters 73 (2016), 33--40.

Digital Library

[10]

Wolf Kienzle and Ken Hinckley. 2014. LightRing: Always-available 2D Input on Any Surface. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 157--160.

Digital Library

[11]

Gierad Laput, Robert Xiao, and Chris Harrison. 2016. ViBand: High-Fidelity Bio-Acoustic Sensing Using Commodity Smartwatch Accelerometers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 321--333.

Digital Library

[12]

Christian Loclair, Sean Gustafson, and Patrick Baudisch. 2010. PinchWatch: a wearable device for one-handed microinteractions. In Proc. MobileHCI, Vol. 10.

[13]

Kent Lyons, Thad Starner, Daniel Plaisted, James Fusia, Amanda Lyons, Aaron Drew, and E. W. Looney. 2004. Twiddler typing. In Proceedings of the 2004 conference on Human factors in computing systems - CHI ‘04. Association for Computing Machinery (ACM).

[14]

I Scott MacKenzie and Shawn X Zhang. 1997. The immediate usability of Graffiti. In Graphics Interface, Vol. 97. 129--137.

Digital Library

[15]

Adiyan Mujibiya, Xiang Cao, Desney S. Tan, Dan Morris, Shwetak N. Patel, and Jun Rekimoto. 2013. The Sound of Touch: On-body Touch and Gesture Sensing Based on Transdermal Ultrasound Propagation. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (ITS '13). ACM, New York, NY, USA, 189--198.

Digital Library

[16]

Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. FingerIO: Using Active Sonar for Fine-Grained Finger Tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1515--1525.

Digital Library

[17]

T. Scott Saponas, Desney S. Tan, Dan Morris, Ravin Balakrishnan, Jim Turner, and James A. Landay. 2009. Enabling Always-available Input with Muscle-computer Interfaces. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST'09). ACM, New York, NY, USA, 167--176.

Digital Library

[18]

Thad Starner, Steve Mann, Bradley Rhodes, Jeffrey Levine, Jennifer Healey, Dana Kirsch, Rosalind W Picard, and Alex Pentland. 1997. Augmented reality through wearable computing. Presence: Teleoperators and Virtual Environments 6, 4 (1997), 386--398.

Digital Library

[19]

Gineke A ten Holt, Marcel JT Reinders, and EA Hendriks. 2007. Multi-dimensional dynamic time warping for gesture recognition. In Thirteenth annual conference of the Advanced School for Computing and Imaging, Vol. 300.

[20]

Hsin-Ruey Tsai, Min-Chieh Hsiu, Jui-Chun Hsiao, Lee-Ting Huang, Mike Chen, and Yi-Ping Hung. 2016. TouchRing: Subtle and Always-available Input Using a Multi-touch Ring. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). ACM, New York, NY, USA, 891--898.

Digital Library

[21]

Sang Ho Yoon, Yunbo Zhang, Ke Huo, and Karthik Ramani. 2016. TRing: Instant and Customizable Interactions with Objects Using an Embedded Magnet and a Finger-Worn Device. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 169--181.

Digital Library

[22]

Cheng Zhang, AbdelKareem Bedri, Gabriel Reyes, Bailey Bercik, Omer T. Inan, Thad E. Starner, and Gregory D. Abowd. 2016. TapSkin: Recognizing On-Skin Input for Smartwatches. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16). ACM, New York, NY, USA, 13--22.

Digital Library

[23]

Cheng Zhang, Xiaoxuan Wang, Anandghan Waghmare, Sumeet Jain, Thomas Ploetz, Omer Inan, Thad E. Starner, and Gregory D. Abowd. 2017. FingOrbits: Interaction with Wearables Using Synchronized Thumb Movements. In Proceedings of the 2017 ACM International Symposium on Wearable Computers (ISWC '17). ACM, New York, NY, USA.

Digital Library

[24]

Cheng Zhang, Qiuyue Xue, Anandghan Waghmare, Sumeet Jain, Yiming Pu, Sinan Hersek, Kent Lyons, Kenneth A. Cunefare, Omer T. Inan, and Gregory D. Abowd. 2017. SoundTrak: Continuous 3D Tracking of a Finger Using Active Acoustics. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 2, Article 30 (June 2017), 25 pages.

Digital Library

[25]

Cheng Zhang, Junrui Yang, Caleb Southern, Thad E. Starner, and Gregory D. Abowd. 2016. WatchOut: Extending Interactions on a Smartwatch with Inertial Sensing. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, New York, NY, USA, 136--143.

Digital Library

[26]

Yang Zhang, Junhan Zhou, Gierad Laput, and Chris Harrison. 2016. SkinTrack: Using the Body As an Electrical Waveguide for Continuous Finger Tracking on the Skin. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1491--1503.

Digital Library

Cited By

Chaffangeon Caillet AGoguey ANigay L(2025)Microgesture ＋ GraspInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103398195:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103398
Mahmud SParikh VLiang QLi KZhang RAjit AGunda VAgarwal DGuimbretiere FZhang C(2024)ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the BodyProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997528:4(1-32)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699752
Yu THu GZhang RLim HMahmud SLee CLi KAgarwal DNie SOh JGuimbretière FZhang C(2024)Ring-a-Pose: A Ring for Continuous Hand Pose TrackingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997418:4(1-30)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699741
Show More Cited By

Index Terms

FingerSound: Recognizing unistroke thumb gestures using a ring
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input
      2. Text input

Recommendations

CyclopsRing: Enabling Whole-Hand and Context-Aware Interactions Through a Fisheye Ring
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology

This paper presents CyclopsRing, a ring-style fisheye imaging wearable device that can be worn on hand webbings to en- able whole-hand and context-aware interactions. Observing from a central position of the hand through a fisheye perspective, ...
LightRing: always-available 2D input on any surface
UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology

We present LightRing, a wearable sensor in a ring form factor that senses the 2d location of a fingertip on any surface, independent of orientation or material. The device consists of an infrared proximity sensor for measuring finger flexion and a 1-...
FingOrbits: interaction with wearables using synchronized thumb movements
ISWC '17: Proceedings of the 2017 ACM International Symposium on Wearable Computers

We present FingOrbits, a wearable interaction technique using synchronized thumb movements. A thumb-mounted ring with an inertial measurement unit and a contact microphone are used to capture thumb movements when rubbing against the other fingers. ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 1, Issue 3

September 2017

2023 pages

EISSN:2474-9567

DOI:10.1145/3139486

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2017

Accepted: 01 June 2017

Received: 01 February 2017

Published in IMWUT Volume 1, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
1,882
Total Downloads

Downloads (Last 12 months)144
Downloads (Last 6 weeks)17

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chaffangeon Caillet AGoguey ANigay L(2025)Microgesture ＋ GraspInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103398195:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103398
Mahmud SParikh VLiang QLi KZhang RAjit AGunda VAgarwal DGuimbretiere FZhang C(2024)ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the BodyProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997528:4(1-32)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699752
Yu THu GZhang RLim HMahmud SLee CLi KAgarwal DNie SOh JGuimbretière FZhang C(2024)Ring-a-Pose: A Ring for Continuous Hand Pose TrackingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997418:4(1-30)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699741
Hiraki HKanazawa SMiura TYoshida MMochimaru MRekimoto J(2024)WhisperMask: a noise suppressive mask-type microphone for whisper speechProceedings of the Augmented Humans International Conference 202410.1145/3652920.3652925(1-14)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3652925
Zhang PZhou QLu TGu HGu N(2024)DeLink: An Adversarial Framework for Defending against Cross-site User Identity LinkageACM Transactions on the Web10.1145/364382818:2(1-34)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1145/3643828
Wang XZhang Y(2024)TextureSightProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314137:4(1-27)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631413
Kin KWan CKoh KMarin ACamgöz NZhang YCai YKovalev FBen-Zacharia MHoople SNunes-Ueno MSanchez-Rodriguez MBhargava AWang RSauser EMa S(2024)STMG: A Machine Learning Microgesture Recognition System for Supporting Thumb-Based VR/AR InputProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642702(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642702
Waghmare ABoldu RWhitmire EKienzle W(2023)OptiRing: Low-Resolution Optical Sensing for Subtle Thumb-to-Index Micro-InteractionsProceedings of the 2023 ACM Symposium on Spatial User Interaction10.1145/3607822.3614538(1-13)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3607822.3614538
Martin DLeng ZGemicioglu TWomack JHeath JNeubauer WKwon HPloetz TStarner T(2023)FingerSpeller: Camera-Free Text Entry Using Smart Rings for American Sign Language Fingerspelling RecognitionProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3614491(1-5)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3614491
Kim TKarlson AGupta AGrossman TWu JAbtahi PCollins CGlueck MSurale H(2023)STAR: Smartphone-analogous Typing in Augmented RealityProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606803(1-13)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606803
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents