research-article

Transferable Microgestures Across Hand Posture and Location Constraints: Leveraging the Middle, Ring, and Pinky Fingers

Authors:

Parastoo Abtahi,

Jackson Rushing,

Christopher Collins,

Michael GlueckAuthors Info & Claims

UIST '23: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology

Article No.: 103, Pages 1 - 17

https://doi.org/10.1145/3586183.3606713

Published: 29 October 2023 Publication History

Abstract

Microgestures can enable auxiliary input when the hands are occupied. Although prior work has evaluated the comfort of microgestures performed by the index finger and thumb, these gestures cannot be performed while the fingers are constrained by specific hand locations or postures. As the hand can be freely positioned with no primary posture, partially constrained while forming a pose, or highly constrained while grasping an object at a specific location, we leverage the middle, ring, and pinky fingers to provide additional opportunities for auxiliary input across varying levels of hand constraints. A design space and applications demonstrate how such microgestures can transfer across hand location and posture constraints. An online study evaluated their comfort and effort and a lab study evaluated their use for task-specific microinteractions. The results revealed that many middle finger microgestures were comfortable, and microgestures performed while forming a pose were preferred over baseline techniques.

Supplementary Material

ZIP File (3606713.zip)

Supplements

Download
150.42 MB

References

[1]

Leonardo Angelini, Francesco Carrino, Stefano Carrino, Maurizio Caon, Omar Abou Khaled, Jürgen Baumgartner, Andreas Sonderegger, Denis Lalanne, and Elena Mugellini. 2014. Gesturing on the steering wheel: A user-elicited taxonomy. In Proc. Int. Conf on Automotive User Interfaces and Interactive Vehicular Applications(AutomotiveUI ’14). ACM, New York, NY, USA, 1–8. https://doi.org/10.1145/2667317.2667414

Digital Library

[2]

Daniel L Ashbrook. 2010. Enabling Mobile Microinteractions. Georgia Institute of Technology.

Digital Library

[3]

Richard A Bolt. 1980. “Put-that-there” Voice and gesture at the graphics interface. In Proc. Conf on Computer Graphics and Interactive Techniques. 262–270.

[4]

Edwin Chan, Teddy Seyed, Wolfgang Stuerzlinger, Xing-Dong Yang, and Frank Maurer. 2016. User elicitation on single-hand microgestures. In Proc. SIGCHI Conf. on Human Factors in Computing Systems. ACM, New York, NY, USA, 3403–3414. https://doi.org/10.1145/2858036.2858589

Digital Library

[5]

Liwei Chan, Rong-Hao Liang, Ming-Chang Tsai, Kai-Yin Cheng, Chao-Huai Su, Mike Y. Chen, Wen-Huang Cheng, and Bing-Yu Chen. 2013. FingerPad: Private and subtle interaction using fingertips. In Proc. ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 255–260. https://doi.org/10.1145/2501988.2502016

Digital Library

[6]

Yu-Chun Chen, Chia-Ying Liao, Shuo-wen Hsu, Da-Yuan Huang, and Bing-Yu Chen. 2020. Exploring user defined gestures for ear-based interactions. Proc. ACM Hum.-Comput. Interact. 4, ISS, Article 186 (Nov. 2020), 20 pages. https://doi.org/10.1145/3427314

Digital Library

[7]

Nalin Chhibber, Hemant Bhaskar Surale, Fabrice Matulic, and Daniel Vogel. 2021. Typealike: Near-keyboard hand postures for expanded laptop interaction. Proc. ACM Hum.-Comput. Interact. 5, ISS, Article 486 (nov 2021), 20 pages. https://doi.org/10.1145/3486952

Digital Library

[8]

Bastian Dewitz, Frank Steinicke, and Christian Geiger. 2019. Functional workspace for one-handed tap and swipe microgestures. In Mensch und Computer 2019 - Workshopband. Gesellschaft für Informatik e.V., Bonn. https://doi.org/10.18420/muc2019-ws-440

[9]

Pierre Dragicevic. 2015. HCI Statistics without p-values. Ph. D. Dissertation. Inria.

[10]

Scott FM Duncan, Caitlin E Saracevic, and Ryosuke Kakinoki. 2013. Biomechanics of the hand. Hand Clinics 29, 4 (2013), 483–492.

[11]

Aarthi Easwara Moorthy and Kim-Phuong L Vu. 2015. Privacy concerns for use of voice activated personal assistant in the public space. Int. J. of Human-Computer Interaction 31, 4 (2015), 307–335.

[12]

Thomas Feix, Javier Romero, Heinz-Bodo Schmiedmayer, Aaron M Dollar, and Danica Kragic. 2015. The GRASP taxonomy of human grasp types. IEEE Trans. on Human-Machine Systems 46, 1 (2015), 66–77.

[13]

Charlotte Häger-Ross and Marc H Schieber. 2000. Quantifying the independence of human finger movements: comparisons of digits, hands, and movement frequencies. J. Neuroscience 20, 22 (2000), 8542–8550.

[14]

Benjamin Hatscher and Christian Hansen. 2018. Hand, foot or voice: Alternative input modalities for touchless interaction in the medical domain. In Proc. ACM Int. Conf. Multimodal Interaction. ACM, New York, NY, USA, 145–153.

Digital Library

[15]

Da-Yuan Huang, Liwei Chan, Shuo Yang, Fan Wang, Rong-Hao Liang, De-Nian Yang, Yi-Ping Hung, and Bing-Yu Chen. 2016. DigitSpace: Designing thumb-to-fingers touch interfaces for one-handed and eyes-free interactions. In Proc. SIGCHI Conf. on Human Factors in Computing Systems(CHI ’16). ACM, New York, NY, USA, 1526–1537. https://doi.org/10.1145/2858036.2858483

Digital Library

[16]

Noriko Kamakura, Michiko Matsuo, Harumi Ishii, Fumiko Mitsuboshi, and Yoriko Miura. 1980. Patterns of static prehension in normal hands. Am. J. Occupational Therapy 34, 7 (1980), 437–445.

[17]

Hsin-Liu (Cindy) Kao, Artem Dementyev, Joseph A. Paradiso, and Chris Schmandt. 2015. NailO: Fingernails as an input surface. In Proc. SIGCHI Conf. on Human Factors in Computing Systems(CHI ’15). ACM, New York, NY, USA, 3015–3018. https://doi.org/10.1145/2702123.2702572

Digital Library

[18]

Maria Karam and schraefel m.c.2005. A taxonomy of gestures in human computer interactions. (2005).

[19]

David Kim, Otmar Hilliges, Shahram Izadi, Alex D. Butler, Jiawen Chen, Iason Oikonomidis, and Patrick Olivier. 2012. Digits: Freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In Proc. ACM Symposium on User Interface Software and Technology(UIST ’12). ACM, New York, NY, USA, 167–176. https://doi.org/10.1145/2380116.2380139

Digital Library

[20]

JMF Landsmeer. 1962. Power grip and precision handling. Annals of the Rheumatic Diseases 21, 2 (1962), 164.

[21]

Chen Liang, Chun Yu, Yue Qin, Yuntao Wang, and Yuanchun Shi. 2021. DualRing: Enabling subtle and expressive hand interaction with dual IMU rings. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 3, Article 115 (Sept. 2021), 27 pages. https://doi.org/10.1145/3478114

Digital Library

[22]

Mingyu Liu, Mathieu Nancel, and Daniel Vogel. 2015. Gunslinger: Subtle arms-down mid-Air interaction. In Proc. ACM Symposium on User Interface Software and Technology(UIST ’15). ACM, New York, NY, USA, 63–71. https://doi.org/10.1145/2807442.2807489

Digital Library

[23]

Christian Loclair, Sean Gustafson, and Patrick Baudisch. 2010. PinchWatch: A wearable device for one-handed microinteractions. In Proc. MobileHCI, Vol. 10. ACM.

[24]

Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. Proc. ACM Human-Computer Interaction 3, CSCW (2019), 1–23.

Digital Library

[25]

Pranav Mistry, Pattie Maes, and Liyan Chang. 2009. WUW-wear Ur world: A wearable gestural interface. In Proc. CHI’09 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, NY, USA, 4111–4116.

Digital Library

[26]

John R Napier. 1956. The prehensile movements of the human hand. J. Bone and Joint Surgery. British volume 38, 4 (1956), 902–913.

[27]

David N Perkins and Gavriel Salomon. 1992. Transfer of learning. Int. Encyclopedia of Education 2 (1992), 6452–6457.

[28]

Timothy Ryan. 2020. Fraudulent responses on Amazon Mechanical Turk: A Fresh Cautionary Tale. https://timryan.web.unc.edu/2020/12/22/fraudulent-responses-on-amazon-mechanical-turk-a-fresh-cautionary-tale/

[29]

Adwait Sharma, Michael A. Hedderich, Divyanshu Bhardwaj, Bruno Fruchard, Jess McIntosh, Aditya Shekhar Nittala, Dietrich Klakow, Daniel Ashbrook, and Jürgen Steimle. 2021. SoloFinger: Robust microgestures while grasping everyday objects. In Proc. SIGCHI Conf. on Human Factors in Computing Systems(CHI ’21). ACM, New York, NY, USA, Article 744, 15 pages. https://doi.org/10.1145/3411764.3445197

Digital Library

[30]

Adwait Sharma, Joan Sol Roo, and Jürgen Steimle. 2019. Grasping Microgestures: Eliciting Single-Hand Microgestures for Handheld Objects. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300632

Digital Library

[31]

Adwait Sharma, Christina Salchow-Hömmen, Vimal Suresh Mollyn, Aditya Shekhar Nittala, Michael A. Hedderich, Marion Koelle, Thomas Seel, and Jürgen Steimle. 2022. SparseIMU: Computational design of sparse IMU layouts for sensing fine-grained finger microgestures. ACM Trans. Comput.-Hum. Interact. (Oct. 2022). https://doi.org/10.1145/3569894

Digital Library

[32]

Mohamed Soliman, Franziska Mueller, Lena Hegemann, Joan Sol Roo, Christian Theobalt, and Jürgen Steimle. 2018. FingerInput: Capturing expressive single-hand thumb-to-finger microgestures. In Proc. International Conference on Interactive Surfaces and Spaces(ISS ’18). ACM, New York, NY, USA, 177–187. https://doi.org/10.1145/3279778.3279799

Digital Library

[33]

Srinath Sridhar, Anna Maria Feit, Christian Theobalt, and Antti Oulasvirta. 2015. Investigating the dexterity of multi-finger input for mid-air text entry(CHI ’15). ACM, New York, NY, USA, 3643–3652. https://doi.org/10.1145/2702123.2702136

Digital Library

[34]

Wei Sun, Franklin Mingzhe Li, Congshu Huang, Zhenyu Lei, Benjamin Steeper, Songyun Tao, Feng Tian, and Cheng Zhang. 2021. ThumbTrak: Recognizing micro-finger poses using a ring with proximity sensing. In Proc. Int. Conf. Mobile Human-Computer Interaction (Toulouse & Virtual, France) (MobileHCI ’21). ACM, New York, NY, USA, Article 2, 9 pages. https://doi.org/10.1145/3447526.3472060

Digital Library

[35]

Hemant Bhaskar Surale, Fabrice Matulic, and Daniel Vogel. 2019. Experimental analysis of barehand mid-air mode-switching techniques in virtual reality. In Proc. SIGCHI Conf. on Human Factors in Computing Systems(CHI ’19). ACM, New York, NY, USA, 14 pages. https://doi.org/10.1145/3290605.3300426

Digital Library

[36]

Yanke Tan, Sang Ho Yoon, and Karthik Ramani. 2017. BikeGesture: User elicitation and performance of micro hand gesture as input for cycling. In Proc. CHI Conf. Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA ’17). ACM, New York, NY, USA, 2147–2154. https://doi.org/10.1145/3027063.3053075

Digital Library

[37]

Hsin-Ruey Tsai, Cheng-Yuan Wu, Lee-Ting Huang, and Yi-Ping Hung. 2016. ThumbRing: Private interactions using one-handed thumb motion input on finger segments. In Proc. Int. Conf. on Human-Computer Interaction with Mobile Devices and Services Adjunct (Florence, Italy) (MobileHCI ’16). ACM, New York, NY, USA, 791–798. https://doi.org/10.1145/2957265.2961859

Digital Library

[38]

Daniel Vogel and Ravin Balakrishnan. 2005. Distant freehand pointing and clicking on very large, high resolution displays. In Proc. ACM Symposium on User Interface Software and Technology(UIST ’05). ACM, New York, NY, USA, 33–42.

Digital Library

[39]

Jérémy Wambecke, Alix Goguey, Laurence Nigay, Lauren Dargent, Daniel Hauret, Stéphanie Lafon, and Jean-Samuel Louis de Visme. 2021. M[Eye]Cro: Eye-Gaze+microgestures for multitasking and interruptions. Proc. ACM Hum.-Comput. Interact. 5, EICS, Article 210 (May 2021), 22 pages. https://doi.org/10.1145/3461732

Digital Library

[40]

Katrin Wolf, Sven Mayer, and Stephan Meyer. 2016. Microgesture detection for remote interaction with mobile devices. In Proc. Int. Conf. on Human-Computer Interaction with Mobile Devices and Services Adjunct(MobileHCI ’16). ACM, New York, NY, USA, 783–790. https://doi.org/10.1145/2957265.2961865

Digital Library

[41]

Katrin Wolf, Anja Naumann, Michael Rohs, and Jörg Müller. 2011. A taxonomy of microinteractions: Defining microgestures based on ergonomic and scenario-Dependent requirements. In IFIP Conf. on Human-Computer Interaction. Springer, 559–575.

[42]

Haijun Xia, Michael Glueck, Michelle Annett, Michael Wang, and Daniel Wigdor. 2022. Iteratively designing gesture vocabularies: A survey and analysis of best practices in the HCI literature. ACM Trans. Comput.-Hum. Interact. 29, 4, Article 37 (May 2022), 54 pages. https://doi.org/10.1145/3503537

Digital Library

[43]

Yen-Ting Yeh, Fabrice Matulic, and Daniel Vogel. 2023. Phone sleight of hand: Finger-based dexterous gestures for physical interaction with mobile phones. In Proc. SIGCHI Conf. on Human Factors in Computing Systems. ACM, New York, NY, USA, 19 pages.

Digital Library

[44]

Jingjie Zheng and Daniel Vogel. 2016. Finger-aware shortcuts. In Proc. SIGCHI Conf. on Human Factors in Computing Systems(CHI ’16). ACM, New York, NY, USA, 4274–4285. https://doi.org/10.1145/2858036.2858355

Digital Library

[45]

Thomas G Zimmerman, Joshua R Smith, Joseph A Paradiso, David Allport, and Neil Gershenfeld. 1995. Applying electric field sensing to human-computer interfaces. In Proc. SIGCHI Conf. on Human Factors in Computing Systems. 280–287.

Digital Library

Cited By

Aponte ACaetano ALuo YSra M(2024)GraV: Grasp Volume Data for the Design of One-Handed XR InterfacesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661567(151-167)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661567
Sharma AIvanov ALai FGrossman TSantosa S(2024)GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of GraspingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661551(1275-1289)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661551
Kin KWan CKoh KMarin ACamgöz NZhang YCai YKovalev FBen-Zacharia MHoople SNunes-Ueno MSanchez-Rodriguez MBhargava AWang RSauser EMa S(2024)STMG: A Machine Learning Microgesture Recognition System for Supporting Thumb-Based VR/AR InputProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642702(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642702

Index Terms

Transferable Microgestures Across Hand Posture and Location Constraints: Leveraging the Middle, Ring, and Pinky Fingers
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input

Recommendations

User Elicitation on Single-hand Microgestures
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Gestural interaction has become increasingly popular, as enabling technologies continue to transition from research to retail. The mobility of miniaturized (and invisible) technologies introduces new uses for gesture recognition. This paper investigates ...
SequenceSense: A Tool for Designing Usable Foot-Based Gestures Using a Sequence-Based Gesture Recognizer
Abstract
Foot-based gestures enable people to interact with mobile and wearable devices when their hands are unavailable for interaction. For the foot gestures to be truly usable, the gestures should be recognizable by the system without being ...
Highlights
- Eliminates multiple data collection studies to design and build usable foot-gestures.
Grasping Microgestures: Eliciting Single-hand Microgestures for Handheld Objects
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Single-hand microgestures have been recognized for their potential to support direct and subtle interactions. While pioneering work has investigated sensing techniques and presented first sets of intuitive gestures, we still lack a systematic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '23: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology

October 2023

1825 pages

ISBN:9798400701320

DOI:10.1145/3586183

Editors:
Sean Follmer
Stanford University, USA
,
Jeff Han,
Jürgen Steimle
Saarland University, Germany
,
Nathalie Henry Riche
Microsoft Research, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

UIST '23

Sponsor:

UIST '23: The 36th Annual ACM Symposium on User Interface Software and Technology

October 29 - November 1, 2023

CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
735
Total Downloads

Downloads (Last 12 months)735
Downloads (Last 6 weeks)65

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Aponte ACaetano ALuo YSra M(2024)GraV: Grasp Volume Data for the Design of One-Handed XR InterfacesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661567(151-167)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661567
Sharma AIvanov ALai FGrossman TSantosa S(2024)GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of GraspingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661551(1275-1289)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661551
Kin KWan CKoh KMarin ACamgöz NZhang YCai YKovalev FBen-Zacharia MHoople SNunes-Ueno MSanchez-Rodriguez MBhargava AWang RSauser EMa S(2024)STMG: A Machine Learning Microgesture Recognition System for Supporting Thumb-Based VR/AR InputProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642702(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642702

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents