research-article

Touchsense: classifying finger touches and measuring their force with an electromyography armband

Authors:

Vincent Becker,

Pietro Oldrati,

Liliana Barrios,

Gábor SörösAuthors Info & Claims

ISWC '18: Proceedings of the 2018 ACM International Symposium on Wearable Computers

Pages 1 - 8

https://doi.org/10.1145/3267242.3267250

Published: 08 October 2018 Publication History

Abstract

Identifying the finger used for touching and measuring the force of the touch provides valuable information on manual interactions. This information can be inferred from electromyography (EMG) of the forearm, measuring the activation of the muscles controlling the hand and fingers. We present Touch-Sense, which classifies the finger touches using a novel neural network architecture and estimates their force on a smartphone in real time based on data recorded from the sensors of an inexpensive and wireless EMG armband. Using data collected from 18 participants with force ground truth, we evaluate our system's performance and limitations. Our system could allow for new interaction paradigms with appliances and objects, which we exemplarily showcase in four applications.

Supplementary Material

ZIP File (p1-becker.zip)

Supplemental files.

Download
585.23 KB

MP4 File (p1-becker.mp4)

Download
131.36 MB

References

[1]

C. Amma, T. Krings, J. Böer, and T. Schultz. Advancing Muscle-Computer Interfaces with High-Density Electromyography. In Proc. CHI '15.

Digital Library

[2]

H. Benko, T. Saponas, D. Morris, and D. Tan. Enhancing Input on and Above the Interactive Surface with Muscle Sensing. In Proc. ITS '09.

Digital Library

[3]

X. Chen, X. Zhang, Z. Zhao, J. Yang, V. Lantz, and K. Q. Wang. Hand Gesture Recognition Research Based on Surface EMG Sensors and 2D-accelerometers. In Proc. ISWC '07.

Digital Library

[4]

A. Colley and J. Häkkilä. Exploring Finger Specific Touch Screen Interaction for Mobile Phone User Interfaces. In Proc. OzCHI '14.

Digital Library

[5]

A. DiDomenico and M. Nussbaum. 2008. Estimation of forces exerted by the fingers using standardised surface electromyography from the forearm. Ergonomics 51, 6 (2008).

[6]

I. Donovan, J. Puchin, K. Okada, and X. Zhang. Simple Space-Domain Features for Low-Resolution sEMG Pattern Recognition. In Proc. EMBC '17.

[7]

Y. Du, W. Jin, W. Wei, Y. Hu, and W. Geng. 2017. Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation. Sensors 17, 3 (2017).

[8]

Y. Du, Y. Wong, W. Jin, W. Wei, Y. Hu, M. Kankanhalli, and W. Geng. Semi-supervised Learning for Surface EMG-based Gesture Recognition. In Proc. IJCAI '17.

Digital Library

[9]

P. Ehrlich and A. Ehrlich. 2008. The Dominant Animal.

[10]

K. Englehart and B. Hudgins. 2003. A Robust, Real-time Control Scheme for Multifunction Myoelectric Control. IEEE Trans. Biomedical Engineering 50, 7 (2003).

[11]

M. Fukumoto and Y. Suenaga. "FingeRing": A Full-time Wearable Interface. In Proc. CHI '94.

Digital Library

[12]

A. Gupta and R. Balakrishnan. DualKey: Miniature Screen Text Entry via Finger Identification. In Proc. CHI '16.

Digital Library

[13]

C. Holz and P. Baudisch. Fiberio: A Touchscreen that Senses Fingerprints. In Proc. UIST '13.

Digital Library

[14]

F. Kerber, M. Puhl, and A. Krüger. User-independent Real-time Hand Gesture Recognition Based on Surface Electromyography. In Proc. MobileHCI '17.

Digital Library

[15]

S. Malik, A. Ranjan, and R. Balakrishnan. Interacting with Large Displays from a Distance with Vision-tracked Multi-finger Gestural Input. In Proc. UIST '05.

Digital Library

[16]

N. Marquardt, J. Kiemer, D. Ledo, S. Boring, and S. Greenberg. Designing User-, Hand-, and Handpart-aware Tabletop Interactions with the TouchID Toolkit. In Proc. ITS '11.

Digital Library

[17]

D. Masson, A. Goguey, S. Malacria, and G. Casiez. WhichFingers: Identifying Fingers on Touch Surfaces and Keyboards Using Vibration Sensors. In Proc. UIST '17.

Digital Library

[18]

J. McIntosh, C. McNeill, M. Fraser, F. Kerber, M. Löchtefeld, and A. Krüger. EMPress: Practical Hand Gesture Classification with Wrist-Mounted EMG and Pressure Sensing. In Proc. CHI '16.

Digital Library

[19]

S. Murugappan, Vinayak, N. Elmqvist, and K. Ramani. Extended Multitouch: Recovering Touch Posture and Differentiating Users Using a Depth Camera. In Proc. UIST '12.

Digital Library

[20]

A. Samadani and D. Kulic. Hand Gesture Recognition based on Surface Electromyography. In Proc. EMBC '14.

[21]

T. Saponas, D. Tan, D. Morris, and R. Balakrishnan. Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-computer Interfaces. In Proc. CHI '08.

Digital Library

[22]

T. Saponas, D. Tan, D. Morris, J. Turner, and J. Landay. Making Muscle-computer Interfaces More Practical. In Proc. CHI '10.

Digital Library

[23]

E. Scheme and K. Englehart. On the Robustness of EMG Features for Pattern Recognition based Myoelectric Control; A Multi-dataset Comparison. In Proc. EMBC '14.

[24]

K. Vega and H. Fuks. Beauty Tech Nails: Interactive Technology at Your Fingertips. In Proc. TEI '14.

Digital Library

[25]

X. Zhang, X. Chen, Y. Li, V. Lantz, K. Wang, and J. Yang. 2011. A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors. IEEE Trans. on Systems, Man, and Cybernetics 41, 6 (2011).

Digital Library

[26]

J. Zheng and D. Vogel. 2016. Finger-Aware Shortcuts. In Proc. CHI '16.

Digital Library

Cited By

Valayil TTanev T(2024)Optimal Design of a Bilateral Stand-Alone Robotic Motion-Assisted Finger Exoskeleton for Home RehabilitationMachines10.3390/machines1210068512:10(685)Online publication date: 29-Sep-2024
https://doi.org/10.3390/machines12100685
Huang ZGao CWang HDeng XLai YMa CQin SLiu YWang H(2024)SpeciFingersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435598:1(1-28)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643559
Zhang YChen KSun Q(2023)Toward Optimized VR/AR Ergonomics: Modeling and Predicting User Neck Muscle ContractionACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591495(1-12)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1145/3588432.3591495
Show More Cited By

Recommendations

TouchSense: Classifying and Measuring the Force of Finger Touches with an Electromyography Armband
AH '18: Proceedings of the 9th Augmented Human International Conference

We present TouchSense, a system to classify and to compute the force of finger touches using an inexpensive, off-the-shelf electromyography (EMG) armband. From EMG input only, we classify the finger touches and estimate the force applied when pressing ...
Implementation of Rehabilitation Exercise Posture Determination System Based on CNN Using EMG and Acceleration Sensors
Intelligent Human Computer Interaction
Abstract
A number of musculoskeletal disorders occur worldwide in occupations that perform physically demanding tasks. In order to treat musculoskeletal disorders, rehabilitation must be performed, but it is not easy to correctly perform rehabilitation ...
Grasp Intent Detection Using Multi Sensorial Data
Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Anthropometry, Human Behavior, and Communication
Abstract
Upper Limbs are expected to have precise functionality to do normal activities and perform a specific occupation. The functionality loss will impair the performance accuracy of the tasks. The amputation of a limb can lead to a great reduction in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISWC '18: Proceedings of the 2018 ACM International Symposium on Wearable Computers

October 2018

307 pages

ISBN:9781450359672

DOI:10.1145/3267242

General Chairs:
Kai Kunze
Keio University
,
Rajesh K. Balan
Singapore Management University
,
Youngki Lee
Singapore Management University
,
Program Chairs:
Michael Beigl
Karlsruhe Institute of Technology
,
Roshan Peiris
Keio University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UbiComp '18

Sponsor:

UbiComp '18: The 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing

October 8 - 12, 2018

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 38 of 196 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
646
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)7

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Valayil TTanev T(2024)Optimal Design of a Bilateral Stand-Alone Robotic Motion-Assisted Finger Exoskeleton for Home RehabilitationMachines10.3390/machines1210068512:10(685)Online publication date: 29-Sep-2024
https://doi.org/10.3390/machines12100685
Huang ZGao CWang HDeng XLai YMa CQin SLiu YWang H(2024)SpeciFingersProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435598:1(1-28)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643559
Zhang YChen KSun Q(2023)Toward Optimized VR/AR Ergonomics: Modeling and Predicting User Neck Muscle ContractionACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591495(1-12)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1145/3588432.3591495
Niijima AKubo Y(2023)Assisting with Fingertip Force Control by Active Bio-Acoustic Sensing and Electrical Muscle StimulationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581192(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581192
Eddy EScheme EBateman S(2023)A Framework and Call to Action for the Future Development of EMG-Based Input in HCIProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580962(1-23)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580962
Kim HYoon BOh SWoo W(2023)Visualizing Hand Force with Wearable Muscle Sensing for Enhanced Mixed Reality Remote CollaborationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332021029:11(4611-4621)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3320210
Zhang DLiao ZXie WWu XXie HXiao JJiang L(2023)Fine-Grained and Real-Time Gesture Recognition by Using IMU SensorsIEEE Transactions on Mobile Computing10.1109/TMC.2021.312047522:4(2177-2189)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TMC.2021.3120475
Nozaki HKataoka YArzate Cruz CShibata FKimura A(2023)Analysis and Considerations of the Controllability of EMG-Based Force InputHuman-Computer Interaction10.1007/978-3-031-35596-7_36(563-572)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35596-7_36
Zhang YLiang BChen BTorrens PAtashzar SLin DSun Q(2022)Force-Aware Interface via Electromyography for Natural VR/AR InteractionACM Transactions on Graphics10.1145/3550454.355546141:6(1-18)Online publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1145/3550454.3555461
Cofer SChen TYang JFollmer S(2022)Detecting Touch and Grasp Gestures Using a Wrist-Worn Optical and Inertial Sensing NetworkIEEE Robotics and Automation Letters10.1109/LRA.2022.31911737:4(10842-10849)Online publication date: Oct-2022
https://doi.org/10.1109/LRA.2022.3191173
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents