Co-Verbal Touch: Enriching Video Telecommunications with Remote Touch Technology
Pages 186 - 194
Abstract
This paper explores the psycho-physiological impact of remote touch in conjunction with conversations shared over video telecommunication. Interlocutors sent upper arm squeezes to each other through paired touch input and output devices in real time. Committed couples were recruited to use the devices in semi-structured discussions after they watched a video clip that contained emotionally charged moments. Users were not told how to use the devices and were free to define the purpose of their use. We examined how remote touch may impact skin conductance and heart rate variability during emotionally charged conversations. Our results revealed that irrespective of speech sentiment, co-verbal touch reduced arousal when compared to discourse without touch. However, touch was used more frequently in utterances that were judged negative and its reduction in arousal was more pronounced than co-verbal touch in positive utterances. These findings suggest that remote touch can be used to attenuate negative conversations or similar circumstances.
References
[1]
U Rajendra Acharya, K Paul Joseph, Natarajan Kannathal, Choo Min Lim, and Jasjit S Suri, 'Heart Rate Variability: A Review', Medical and biological engineering and computing, 44 (2006), 1031-51.
[2]
Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf, 'Flair: An Easy-to-Use Framework for State-of-the-Art Nlp', (2019), pp. 54-59.
[3]
Bradley M Appelhans, and Linda J Luecken, 'Heart Rate Variability as an Index of Regulated Emotional Responding', Review of general psychology, 10 (2006), 229-40.
[4]
Gary G Berntson, J Thomas Bigger Jr, Dwain L Eckberg, Paul Grossman, Peter G Kaufmann, Marek Malik, Haikady N Nagaraja, Stephen W Porges, J Philip Saul, and Peter H Stone, 'Heart Rate Variability: Origins, Methods, and Interpretive Caveats', Psychophysiology, 34 (1997), 623-48.
[5]
Emily A Butler, Frank H Wilhelm, and James J Gross, 'Respiratory Sinus Arrhythmia, Emotion, and Emotion Regulation During Social Interaction', Psychophysiology, 43 (2006), 612-22.
[6]
Angela Chan, Francis Quek, Haard Panchal, Joshua Howell, Takashi Yamauchi, and Jinsil Hwaryoung Seo, 'The Effect of Co-Verbal Remote Touch on Electrodermal Activity and Emotional Response in Dyadic Discourse', Sensors, 21 (2021), 168.
[7]
Angela Chan, Niloofar Zarei, Takashi Yamauchi, Jinsil Seo, and Francis Quek, 'Touch Media: Investigating the Effects of Remote Touch on Music-Based Emotion Elicitation', in 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) (2019), pp. 1-7.
[8]
Adrian David Cheok, and Emma Yann Zhang, 'Kissenger: Transmitting Kiss through the Internet', in Human–Robot Intimate Relationships, ed. by Adrian David Cheok and Emma Yann Zhang (Cham: Springer International Publishing, 2019), pp. 77-97.
[9]
Adrian David Cheok, and Emma Yann Zhang, 'Huggy Pajama: Remote Hug System for Family Communication', in Human–Robot Intimate Relationships (Springer, 2019), pp. 33-75.
[10]
1Adrian David Cheok, and Emma Yann Zhang 'Electrical Machine for Remote Kissing, and Engineering Measurement of Its Remote Communication Effects, Including Modified Turing Test', Journal of Future Robot Life (2020), 1-24.
[11]
Anthony B Ciccone, Jacob A Siedlik, Jill M Wecht, Jake A Deckert, Nhuquynh D Nguyen, and Joseph P Weir, 'Reminder: Rmssd and Sd1 Are Identical Heart Rate Variability Metrics', Muscle & nerve, 56 (2017), 674-78.
[12]
Johann Citérin, and Abderrahmane Kheddar, 'Electro-Active Polymer Actuators for Tactile Displays', in The Sense of Touch and Its Rendering (Springer, 2008), pp. 131-54.
[13]
H. J. Conradi, A. Noordhof, and A. Arntz, 'Improvement of Conflict Handling: Hand-Holding During and after Conflict Discussions Affects Heart Rate, Mood, and Observed Communication Behavior in Romantic Partners', Journal of Sex & Marital Therapy, 46 (2020), 419-34.
[14]
Birgül Vural Doğru, Hediye Utli, and Fisun Şenuzun Aykar, 'Effect of Therapeutic Touch on Daytime Sleepiness, Stress and Fatigue among Students of Nursing and Midwifery: A Randomized Sham-Controlled Trial', Complementary Therapies in Clinical Practice, 43 (2021), 101322.
[15]
M. A. Eid, and H. Al Osman, 'Affective Haptics: Current Research and Future Directions', IEEE Access, 4 (2016), 26-40.
[16]
Empatica, 'Emptica E4 Wristband'2015) <https://www.empatica.com/e4-wristband>[Accessed November 12 2015].
[17]
Roberta Etzi, Carlotta Carta, and Alberto Gallace, 'Stroking and Tapping the Skin: Behavioral and Electrodermal Effects', Experimental Brain Research, 236 (2018), 453-61.
[18]
Ruth Feldman, Magi Singer, and Orna Zagoory, 'Touch Attenuates Infants’ Physiological Reactivity to Stress', Developmental science, 13 (2010), 271-78.
[19]
Tiffany Field, 'Social Touch, Ct Touch and Massage Therapy: A Narrative Review', Dev. Rev., 51 (2019), 123-45.
[20]
Arnstein Finset, Tonje L. Stensrud, Erik Holt, William Verheul, and Jozien Bensing, 'Electrodermal Activity in Response to Empathic Statements in Clinical Interviews with Fibromyalgia Patients', Patient education and counseling, 82 (2011), 355-60.
[21]
C. H. E. Gilbert, and Erric Hutto, 'Vader: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text', in Eighth International Conference on Weblogs and Social Media (ICWSM-14). Available at (20/04/16) http://comp. social. gatech. edu/papers/icwsm14. vader. hutto. pdf (2014), p. 82.
[22]
Daniel Gooch, and Leon Watts, 'Yourgloves, Hothands and Hotmits: Devices to Hold Hands at a Distance', in Proceedings of the 25th annual ACM symposium on User Interface Software and Technology (2012), pp. 157-66.
[23]
Karen M. Grewen, Bobbi J. Anderson, Susan S. Girdler, and Kathleen C. Light, 'Warm Partner Contact Is Related to Lower Cardiovascular Reactivity', Behavioral Medicine, 29 (2003), 123-30.
[24]
James J. Gross, 'Antecedent-and Response-Focused Emotion Regulation: Divergent Consequences for Experience, Expression, and Physiology', Journal of personality and social psychology, 74 (1998), 224.
[25]
James J. Gross, and Robert W. Levenson, 'Emotional Suppression: Physiology, Self-Report, and Expressive Behavior', Journal of personality and social psychology, 64 (1993), 970.
[26]
Barbara Grosz, and Candace L. Sidner, 'Attention, Intentions, and the Structure of Discourse', Comput. Linguist. (1986).
[27]
Antal Haans, and Wijnand A IJsselsteijn, 'The Virtual Midas Touch: Helping Behavior after a Mediated Social Touch', IEEE Transactions on Haptics, 2 (2009), 136-40.
[28]
Sohyun C. Han, Hannah L. Schacter, Adela C. Timmons, Yehsong Kim, Stassja Sichko, Corey Pettit, Theodora Chaspari, Shrikanth Narayanan, and Gayla Margolin, 'Romantic Partner Presence and Physiological Responses in Daily Life: Attachment Style as a Moderator', Biological Psychology, 161 (2021), 108082.
[29]
Matthew J Hertenstein, Dacher Keltner, Betsy App, Brittany A Bulleit, and Ariane R Jaskolka, 'Touch Communicates Distinct Emotions', Emotion, 6 (2006), 528.
[30]
IBM, 'Watson Tone Analyzer' <https://www.ibm.com/cloud/watson-tone-analyzer>.
[31]
Mirja Ilves, and Veikko Surakka, 'Heart Rate Responses to Synthesized Affective Spoken Words', Advances in Human-Computer Interaction, 2012 (2012).
[32]
Neel Kant, Raul Puri, Nikolai Yakovenko, and Bryan Catanzaro, 'Practical Text Classification with Large Pre-Trained Language Models', arXiv [cs.CL] (2018).
[33]
Kun-chan Lan, Paweeya Raknim, Wei-Fong Kao, and Jyh-How Huang, 'Toward Hypertension Prediction Based on Ppg-Derived Hrv Signals: A Feasibility Study', Journal of medical systems, 42 (2018), 1-7.
[34]
Ledalab, 'Ledalab' <http://www.ledalab.de/>.
[35]
Jessica P Lougheed, and Tom Hollenstein, 'Arousal Transmission and Attenuation in Mother–Daughter Dyads During Adolescence', Social Development, 27 (2018), 19-33.
[36]
Kym Maclaren, 'Touching Matters: Embodiments of Intimacy', Emotion, Space and Society, 13 (2014), 95-102.
[37]
Lucrezia Mangione, Diane Swengros, and Joel G Anderson, 'Mental Health Wellness and Biofield Therapies: An Integrative Review', Issues in mental health nursing, 38 (2017), 930-44.
[38]
F. Mueller, F. Vetere, M. Gibbs, J. Kjedskov, S. Pedell, and S. Howard, 'Hug over a Distance', in CHI' 05 Extended Abstracts (Portland, OR: 2005).
[39]
Hideyuki Nakanishi, Kazuaki Tanaka, and Yuya Wada, 'Remote Handshaking: Touch Enhances Video-Mediated Social Telepresence', in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Association for Computing Machinery, 2014), pp. 2143-52.
[40]
Minh Hao Nguyen, Jonathan Gruber, Will Marler, Amanda Hunsaker, Jaelle Fuchs, and Eszter Hargittai, 'Staying Connected While Physically Apart: Digital Communication When Face-to-Face Interactions Are Limited', New Media & Society (2021), 1461444820985442.
[41]
Giovanni Novembre, Roberta Etzi, and India Morrison, 'Hedonic Responses to Touch Are Modulated by the Perceived Attractiveness of the Caresser', Neuroscience, 464 (2021), 79-89.
[42]
Katja Paananen, and Riikka Vaununmaa, 'Discussions on Intimate Partner Violence and Electrodermal Activity in Couple Therapy', (2016).
[43]
Helena Päivinen, Juha Holma, Anu Karvonen, Virpi-Liisa Kykyri, Valeri Tsatsishvili, Jukka Kaartinen, Markku Penttonen, and Jaakko Seikkula, 'Affective Arousal During Blaming in Couple Therapy: Combining Analyses of Verbal Discourse and Physiological Responses in Two Case Studies', Contemporary Family Therapy, 38 (2016), 373-84.
[44]
Athina Papadopoulou, Jaclyn Berry, Terry Knight, and Rosalind Picard, 'Affective Sleeve: Wearable Materials with Haptic Action for Promoting Calmness', Distributed, Ambient and Pervasive Interactions (2019), 304-19.
[45]
Jussi Rantala, Katri Salminen, Roope Raisamo, and Veikko Surakka, 'Touch Gestures in Communicating Emotional Intention Via Vibrotactile Stimulation', International Journal of Human-Computer Studies (2013).
[46]
Nicolás Ruiz-Robledillo, and Luis Moya-Albiol, 'Lower Electrodermal Activity to Acute Stress in Caregivers of People with Autism Spectrum Disorder: An Adaptive Habituation to Stress', Journal of autism and developmental disorders, 45 (2015), 576-88.
[47]
Elham Saadatian, Hooman Samani, Rahul Parsani, Anshul Vikram Pandey, Jinhui Li, Lenis Tejada, Adrian David Cheok, and Ryohei Nakatsu, 'Mediating Intimacy in Long-Distance Relationships Using Kiss Messaging', Int. J. Hum. Comput. Stud., 72 (2014), 736-46.
[48]
Marco Sarchiapone, Carla Gramaglia, Miriam Iosue, Vladimir Carli, Laura Mandelli, Alessandro Serretti, Debora Marangon, and Patrizia Zeppegno, 'The Association between Electrodermal Activity (Eda), Depression and Suicidal Behaviour: A Systematic Review and Narrative Synthesis', BMC psychiatry, 18 (2018), 1-27.
[49]
Sara Shahrestani, Elizabeth M Stewart, Daniel S Quintana, Ian B Hickie, and Adam J Guastella, 'Heart Rate Variability During Adolescent and Adult Social Interactions: A Meta-Analysis', Biological psychology, 105 (2015), 43-50.
[50]
Rannveig Sigurvinsdottir, Ingibjorg E Thorisdottir, and Haukur Freyr Gylfason, 'The Impact of Covid-19 on Mental Health: The Role of Locus on Control and Internet Use', International Journal of Environmental Research and Public Health, 17 (2020), 6985.
[51]
Samarth Singhal, Carman Neustaedter, Yee Loong Ooi, Alissa N. Antle, and Brendan Matkin, 'Flex-N-Feel: The Design and Evaluation of Emotive Gloves for Couples to Support Touch over Distance', in Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Association for Computing Machinery, 2017), pp. 98-110.
[52]
Juan Sztajzel, 'Heart Rate Variability: A Noninvasive Electrocardiographic Method to Measure the Autonomic Nervous System', Swiss medical weekly, 134 (2004), 514-22.
[53]
Naomi Takahashi, Akira Kuriyama, Hoshinori Kanazawa, Yoshimitsu Takahashi, and Takeo Nakayama, 'Validity of Spectral Analysis Based on Heart Rate Variability from 1‐Minute or Less Ecg Recordings', Pacing and Clinical Electrophysiology, 40 (2017), 1004-09.
[54]
Chantal Triscoli, Ilona Croy, Susann Steudte-Schmiedgen, Håkan Olausson, and Uta Sailer, 'Heart Rate Variability Is Enhanced by Long-Lasting Pleasant Touch at Ct-Optimized Velocity', Biological Psychology, 128 (2017), 71-81.
[55]
Jan B. F. van Erp, and Alexander Toet, 'Social Touch in Human–Computer Interaction', Frontiers in Digital Humanities, 2 (2015), 2.
[56]
Rongrong Wang, and Francis Quek, 'Touch & Talk: Contextualizing Remote Touch for Affective Interaction', in Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction (Cambridge, Massachusetts, USA: ACM, 2010), pp. 13-20.
[57]
Leo Westerlund, and Jari Kaivo-oja, 'Digital Evolution–from Information Society to Ubiquitous Society', Service Design: On the Evolution of Design Expertise, Lahti University of Applied Sciences Series A, Research reports, Part, 16 (2012), 137-53.
[58]
Graham Wilson, and Stephen A Brewster, 'Multi-Moji: Combining Thermal, Vibrotactile & Visual Stimuli to Expand the Affective Range of Feedback', in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (2017), pp. 1743-55.tion society to ubiquitous society. Service Design: On the Evolution of Design Expertise, Lahti University of Applied Sciences Series A, Research reports, Part 16(2012), 137-153.
Recommendations
To touch or not to touch?: comparing 2D touch and 3D mid-air interaction on stereoscopic tabletop surfaces
SUI '13: Proceedings of the 1st symposium on Spatial user interactionRecent developments in touch and display technologies have laid the groundwork to combine touch-sensitive display systems with stereoscopic three-dimensional (3D) display. Although this combination provides a compelling user experience, interaction with ...
Keep in touch: channel, expectation and experience
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThis paper investigates whether and how digitally mediated social touch (remote touch) may influence the sense of connectedness toward a speaker and the emotional experience of what is being communicated. We employ an 'augmented' storytelling ...
Comments
Information & Contributors
Information
Published In
October 2021
876 pages
ISBN:9781450384810
DOI:10.1145/3462244
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 October 2021
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
ICMI '21
Sponsor:
ICMI '21: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
October 18 - 22, 2021
QC, Montréal, Canada
Acceptance Rates
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 261Total Downloads
- Downloads (Last 12 months)95
- Downloads (Last 6 weeks)24
Reflects downloads up to 14 Sep 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in