Touch Recognition with Attentive End-to-End Model
Pages 694 - 698
Abstract
Touch is the earliest sense to develop and the first mean of contact with the external world. Touch also plays a key role in our socio-emotional communication: we use it to communicate our feelings, elicit strong emotions in others and modulate behavior (e.g compliance). Although its relevance, touch is an understudied modality in Human-Machine-Interaction compared to audition and vision. Most of the social touch recognition systems require a feature engineering step making them difficult to compare and to generalize to other databases. In this paper, we propose an end-to-end approach. We present an attention-based end-to-end model for touch gesture recognition evaluated on two public datasets (CoST and HAART) in the context of the ICMI 15 Social Touch Challenge. Our model gave a similar level of accuracy: 61% for CoST and 68% for HAART and uses self-attention as an alternative to feature engineering and Recurrent Neural Networks.
References
[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (Savannah, GA, USA) (OSDI'16). USENIX Association, USA, 265--283.
[2]
Saad Albawi, Oguz Bayat, Saad Al-Azawi, and Osman N. Ucan. 2018. Social Touch Gesture Recognition Using Convolutional Neural Network. https://doi.org/10.1155/2018/6973103
[3]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv:cs.CL/1409.0473
[4]
Tugce Balli Altuglu and Kerem Altun. 2015. Recognizing Touch Gestures for Social Human-Robot Interaction. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction ICMI '15. ACM Press, Seattle, Washington, USA, 407--413. https: //doi.org/10.1145/2818346.2830600
[5]
James Bergstra and Yoshua Bengio. 2012. Random Search for HyperParameter Optimization. J. Mach. Learn. Res. 13, null (Feb. 2012), 281--305.
[6]
Xi Laura Cang, Paul Bucci, Andrew Strang, Jeff Allen, Karon MacLean, and H.Y. Sean Liu. 2015. Different Strokes and Different Folks: Economical Dynamic Surface Sensing and Affect-Related Touch Recognition. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). Association for Computing Machinery, Seattle, Washington, USA, 147--154. https://doi.org/10.1145/2818346.2820756
[7]
Carissa J. Cascio, David Moore, and Francis McGlone. 2018. Social touch and human development. Developmental Cognitive Neuroscience (April 2018). https://doi.org/10.1016/j.dcn.2018.04.009
[8]
April H. Crusco and Christopher G. Wetzel. 1984. The Midas Touch: The Effects of Interpersonal Touch on Restaurant Tipping. Personality and Social Psychology Bulletin 10, 4 (Dec. 1984), 512--517. https://doi. org/10.1177/0146167284104003
[9]
Quentin Debard, Christian Wolf, Stéphane Canu, and Julien Arné. 2018. Learning to recognize touch gestures: recurrent vs. convolutional features and dynamic sampling. IEEE, 114--121. https://doi.org/10. 1109/FG.2018.00026
[10]
Van Erp, Jan B. F, and Alexander Toet. 2015. Social Touch in Human--Computer Interaction. Frontiers in Digital Humanities 2 (2015). https://doi.org/10.3389/fdigh.2015.00002
[11]
Tiffany Field. 2010. Touch for socioemotional and physical well-being: A review. Developmental Review 30, 4 (Dec. 2010), 367--383. https: //doi.org/10.1016/j.dr.2011.01.001
[12]
Yona Falinie A. Gaus, Temitayo Olugbade, Asim Jan, Rui Qin, Jingxin Liu, Fan Zhang, Hongying Meng, and Nadia Bianchi-Berthouze. 2015. Social Touch Gesture Recognition Using Random Forest and Boosting on Distinct Feature Sets. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). ACM, New York, NY, USA, 399--406. https://doi.org/10.1145/2818346.2830599
[13]
Andrew K. Gulledge, Michelle H. Gulledge, and Robert F. Stahmannn. 2003. Romantic Physical Affection Types and Relationship Satisfaction. The American Journal of Family Therapy 31, 4 (July 2003), 233--242. https://doi.org/10.1080/01926180390201936
[14]
Markus Heinrichs, Thomas Baumgartner, Clemens Kirschbaum, and Ulrike Ehlert. 2003. Social support and oxytocin interact to suppress cortisol and subjective responses to psychosocial stress. Biological Psychiatry 54, 12 (Dec. 2003), 1389--1398.
[15]
Maria Henricson, Anna-Lena Berglund, Sylvia Määttä, Rolf Ekman, and Kerstin Segesten. 2008. The outcome of tactile touch on oxytocin in intensive care patients: a randomised controlled trial. Journal of Clinical Nursing 17, 19 (Oct. 2008), 2624--2633. https://doi.org/10.1111/ j.1365--2702.2008.02324.x
[16]
Matthew J. Hertenstein, Rachel M. Holmes, Margaret McCullough, and Dacher Keltner. 2009. The communication of emotion via touch. Emotion 9, 4 (2009), 566--573. https://doi.org/10.1037/a0016108
[17]
Dana Hughes, Nicholas Farrow, Halley Profita, and Nikolaus Correll. 2015. Detecting and Identifying Tactile Gestures Using Deep Autoencoders, Geometric Moments and Gesture Level Features. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). ACM, New York, NY, USA, 415--422. https://doi.org/10.1145/2818346.2830601
[18]
D. Hughes, A. Krauthammer, and N. Correll. 2017. Recognizing social touch gestures using recurrent and convolutional neural networks. In 2017 IEEE International Conference on Robotics and Automation (ICRA). 2315--2321. https://doi.org/10.1109/ICRA.2017.7989267
[19]
Merel M. Jung, Xi Laura Cang, Mannes Poel, and Karon E. MacLean. 2015. Touch Challenge '15: Recognizing Social Touch Gestures. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). ACM, New York, NY, USA, 387--390. https: //doi.org/10.1145/2818346.2829993
[20]
Merel M. Jung, Mannes Poel, Ronald Poppe, and Dirk K. J. Heylen. 2017. Automatic recognition of touch gestures in the corpus of social touch. Journal on Multimodal User Interfaces 11, 1 (March 2017), 81--96. https://doi.org/10.1007/s12193-016-0232--9
[21]
Merel M. Jung, Ronald Poppe, Mannes Poel, and Dirk K.J. Heylen. 2014. Touching the Void -- Introducing CoST: Corpus of Social Touch. In Proceedings of the 16th International Conference on Multimodal Interaction (ICMI '14). ACM, New York, NY, USA, 120--127. https://doi.org/10.1145/2663204.2663242
[22]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv:cs.LG/1412.6980
[23]
Takanori Shibata, Yukitaka Kawaguchi, and Kazuyoshi Wada. 2012. Investigation on People Living with Seal Robot at Home. International Journal of Social Robotics 4, 1 (Jan. 2012), 53--63. https://doi.org/10. 1007/s12369-011-0111--1
[24]
Hyeon-Jeong Suk, Sang-Hoon Jeong, Tae-Heun Yang, and Dong-Soo Kwon. 2009. TACTILE SENSATION AS EMOTION ELICITOR. KANSEI Engineering International 8, 2 (2009), 153--158. https://doi.org/10.5057/ E081120-ISES06
[25]
Viet Cuong Ta, Wafa Johal, Maxime Portaz, Eric Castelli, and Dominique Vaufreydaz. 2015. The Grenoble System for the Social Touch Challenge at ICMI 2015. https://doi.org/10.1145/2818346.2830598
[26]
Siewart van Wingerden, Tobias J. Uebbing, Merel M. Jung, and Mannes Poel. 2014. A Neural Network Based Approach to Social Touch Classification. In Proceedings of the 2014 Workshop on Emotion Representation and Modelling in Human-Computer-Interaction-Systems (ERM4HCI '14). ACM, New York, NY, USA, 7--12. https://doi.org/10.1145/2668056.
[27]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. https://arxiv.org/pdf/1706.03762.pdf
[28]
Kazuyoshi Wada, Takanori Shibata, Toshimitsu Musha, and Shin Kimura. 2008. Robot therapy for elders affected by dementia. IEEE Engineering in Medicine and Biology Magazine 27, 4 (July 2008), 53--60. https://doi.org/10.1109/MEMB.2008.919496
[29]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical Attention Networks for Document Classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human ICMI '20, October 25--29, 2020, Virtual event, Netherlands EL BANI and Chetouani. Language Technologies. Association for Computational Linguistics, San Diego, California, 1480--1489. https://doi.org/10.18653/v1/N16--1174
[30]
Nan Zhou and Jun Du. 2016. Recognition of Social Touch Gestures Using 3D Convolutional Neural Networks. In Pattern Recognition (Communications in Computer and Information Science), Tieniu Tan, Xuelong Li, Xilin Chen, Jie Zhou, Jian Yang, and Hong Cheng (Eds.). Springer, Singapore, 164--173. https://doi.org/10.1007/978--981--10--3002--4_14
Index Terms
- Touch Recognition with Attentive End-to-End Model
Recommendations
Touching the Void -- Introducing CoST: Corpus of Social Touch
ICMI '14: Proceedings of the 16th International Conference on Multimodal InteractionTouch behavior is of great importance during social interaction. To transfer the tactile modality from interpersonal interaction to other areas such as Human-Robot Interaction (HRI) and remote communication automatic recognition of social touch is ...
Towards Social Touch Intelligence: Developing a Robust System for Automatic Touch Recognition
ICMI '14: Proceedings of the 16th International Conference on Multimodal InteractionTouch behavior is of great importance during social interaction. Automatic recognition of social touch is necessary to transfer the touch modality from interpersonal interaction to other areas such as Human-Robot Interaction (HRI). This paper describes ...
Touch Challenge '15: Recognizing Social Touch Gestures
ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal InteractionAdvances in the field of touch recognition could open up applications for touch-based interaction in areas such as Human-Robot Interaction (HRI). We extended this challenge to the research community working on multimodal interaction with the goal of ...
Comments
Information & Contributors
Information
Published In
October 2020
920 pages
ISBN:9781450375818
DOI:10.1145/3382507
- General Chairs:
- Khiet Truong,
- Dirk Heylen,
- Mary Czerwinski,
- Program Chairs:
- Nadia Berthouze,
- Mohamed Chetouani,
- Mikio Nakano
Copyright © 2020 ACM.
© 2020 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 October 2020
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
ICMI '20
Sponsor:
ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
October 25 - 29, 2020
Virtual Event, Netherlands
Acceptance Rates
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 159Total Downloads
- Downloads (Last 12 months)14
- Downloads (Last 6 weeks)2
Reflects downloads up to 03 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in