Low-latency Classification of Social Haptic Gestures Using Transformers
Pages 137 - 141
Abstract
Social touch, and its recognition and classification, is increasingly important in human-robot interaction. We present a Transformer-based model trained and evaluated on an open-source dataset. The dataset, the Human-Animal Affective Robot Touch (HAART) dataset, was collected for the 2015 Recognition of Touch Gesture Challenge (RTGC 2015) and contains different haptic actions directed at a robotic animal. The actions are recorded using a multi-resolution pressure sensor. We feed the output, containing the touch type to the Nao robot to make the robot sense the touch type. The proposed transformer-based gesture classification model achieved 72.8% classification accuracy in 2.67 seconds, which outperforms the best-submitted algorithm of the RTGC 2015 which has a test classification accuracy of 70.9 % and needed 8 seconds.
References
[1]
Mohammed Ibrahim Ahmed Al-Mashhadani, Theyazn HH Aldhyani, Mosleh Hmoud Al-Adhaileh, Alwi M Bamhdi, Mohammed Yahya Alzahrani, Fawaz Waselallah Alsaade, and Hasan Alkahtani. 2021. Human-Animal Affective Robot Touch Classification Using Deep Neural Network. Comput. Syst. Sci. Eng. 38, 1 (2021), 25--37.
[2]
Saad Albawi, Oguz Bayat, Saad Al-Azawi, and Osman N Ucan. 2018. Social touch gesture recognition using convolutional neural network. Computational Intelligence and Neuroscience 2018 (2018).
[3]
Hasan Alkahtani, Theyazn HH Aldhyani, and Mohammed Al-Yaari. 2020. Adaptive anomaly detection framework model objects in cyberspace. Applied Bionics and Biomechanics 2020 (2020).
[4]
Tugce Balli Altuglu and Kerem Altun. 2015. Recognizing touch gestures for social human-robot interaction. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 407--413.
[5]
Xi Laura Cang, Paul Bucci, Andrew Strang, Jeff Allen, Karon MacLean, and HY Sean Liu. 2015. Different strokes and different folks: Economical dynamic surface sensing and affect-related touch recognition. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 147--154.
[6]
Angela Chan, Francis Quek, Haard Panchal, Joshua Howell, Takashi Yamauchi, and Jinsil Hwaryoung Seo. 2020. The Effect of Co-Verbal Remote Touch on Electrodermal Activity and Emotional Response in Dyadic Discourse. Sensors 21, 1 (2020), 168.
[7]
Jonathan Chang, Karon MacLean, and Steve Yohanan. 2010. Gesture recognition in the haptic creature. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 385--391.
[8]
Anna Flagg and Karon MacLean. 2013. Affective touch gesture recognition for a furry zoomorphic machine. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction. 25--32.
[9]
Yona Falinie A Gaus, Temitayo Olugbade, Asim Jan, Rui Qin, Jingxin Liu, Fan Zhang, Hongying Meng, and Nadia Bianchi-Berthouze. 2015. Social touch gesture recognition using random forest and boosting on distinct feature sets. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 399--406.
[10]
Marcel Heerink, Ben Krose, Vanessa Evers, and Bob Wielinga. 2006. The influence of a robot's social abilities on acceptance by elderly users. In ROMAN 2006-The 15th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 521--526.
[11]
Geoffrey E Hinton and Sam Roweis. 2002. Stochastic neighbor embedding. Advances in neural information processing systems 15 (2002).
[12]
Dana Hughes, Nicholas Farrow, Halley Profita, and Nikolaus Correll. 2015. Detecting and identifying tactile gestures using deep autoencoders, geometric moments and gesture level features. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 415--422.
[13]
Dana Hughes, Alon Krauthammer, and Nikolaus Correll. 2017. Recognizing social touch gestures using recurrent and convolutional neural networks. In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2315--2321.
[14]
Merel M Jung, Xi Laura Cang, Mannes Poel, and Karon E MacLean. 2015. Touch challenge'15: Recognizing social touch gestures. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 387--390.
[15]
Merel M Jung, Lisa Van der Leij, and Saskia M Kelders. 2017. An exploration of the benefits of an animallike robot companion with more advanced touch interaction capabilities for dementia care. Frontiers in ICT 4 (2017), 16.
[16]
Viet-Cuong Ta, Wafa Johal, Maxime Portaz, Eric Castelli, and Dominique Vaufreydaz. 2015. The Grenoble system for the social touch challenge at ICMI 2015. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 391--398.
[17]
Puja Varsani. 2018. The power of affective touch within social robotics. Ph. D. Dissertation. Middlesex University.
[18]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ?ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[19]
Nan Zhou and Jun Du. 2016. Recognition of social touch gestures using 3D convolutional neural networks. In Chinese Conference on Pattern Recognition. Springer, 164--173.
Index Terms
- Low-latency Classification of Social Haptic Gestures Using Transformers
Recommendations
Haptic target acquisition to enable spatial gestures in nonvisual displays
GI '13: Proceedings of Graphics Interface 2013Nonvisual natural user interfaces can facilitate gesture-based interaction without having to rely on a physical display. Consequently, this may significantly increase available interaction space on mobile devices, where screen real estate is limited. ...
Multiclass wound image classification using an ensemble deep CNN-based classifier
AbstractAcute and chronic wounds are a challenge to healthcare systems around the world and affect many people's lives annually. Wound classification is a key step in wound diagnosis that would help clinicians to identify an optimal treatment ...
Highlights- Deep Learning-based methods can be used for wound analysis.
- DCNN-based ...
Eliciting Mid-Air Gestures for Wall-Display Interaction
NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer InteractionFreehand mid-air gestures are a promising input method for interacting with wall displays. However, work on mid-air gestures for wall-display interaction has mainly explored what is technically possible, which might not result in gestures that users ...
Comments
Information & Contributors
Information
Published In
March 2023
612 pages
ISBN:9781450399708
DOI:10.1145/3568294
- General Chairs:
- Ginevra Castellano,
- Laurel Riek,
- Program Chairs:
- Maya Cakmak,
- Iolanda Leite
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 March 2023
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
HRI '23: ACM/IEEE International Conference on Human-Robot Interaction
March 13 - 16, 2023
Stockholm, Sweden
Acceptance Rates
Overall Acceptance Rate 268 of 1,124 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 119Total Downloads
- Downloads (Last 12 months)42
- Downloads (Last 6 weeks)3
Reflects downloads up to 14 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in