skip to main content
10.1145/3462244.3479900acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

A Multimodal Dataset and Evaluation for Feature Estimators of Temporal Phases of Anxiety

Published: 18 October 2021 Publication History

Abstract

Vicious cycles of anxiety responses underlie the onset of increasingly prevalent and highly impairing anxiety disorders and also contribute to their maintenance. Our goal is to evaluate whether different anxiety responses are evident in temporal patterns of physiological and behavioral features. Consequently, we established a rich multimodal-multisensor dataset of cardiac, electrodermal, movement, posture, and speech measures from 95 young adults during two anxiety experiments that induce social anxiety and bug-phobic anxiety. A subset of this dataset is publicly available at “Anxiety Phases Dataset” Figshare repository. We adopted a generalized mixed model approach and found that 10 out of 14 feature trajectories modeled for high- and low-anxiety groups differ significantly at 0.001 level in magnitude, creating at least two temporal phases in both groups. Further differences in magnitude, duration and the number of phases were observed for responses of confrontation, safety behaviors, escape, and avoidance in the high-anxiety group. Our findings contribute to the long-term aim of designing multimodal systems that have great potential to reduce the impacts of anxiety disorders and improve therapy.

Supplementary Material

MP4 File (ICMI21-fp1144.mp4)
This is a brief oral presentation by Hashini Senaratne on the paper titled ``A Multimodal Dataset and Evaluation for Feature Estimators of Temporal Phases of Anxiety''. It starts with the background on vicious cycles of anxiety responses that underlie the development and maintenance of anxiety disorders and gaps in the objective anxiety assessment field. Next, it details its hypotheses related to evaluating whether different anxiety responses are evident in temporal patterns of physiological and behavioral features. Subsequently, it details a newly established multimodal-multisensor dataset and extracted cardiac, electrodermal, movement, posture, and speech features. It also summarizes findings on significantly different feature trajectories for high- and low-anxiety groups and responses of confrontation, safety behaviors, escape, and avoidance, based on a generalized mixed model approach. Some examples demonstrating differences occurred in magnitude, duration, and the number of phases are also discussed.

References

[1]
Ahmet Akin and Bayram Çetin. 2007. The Depression Anxiety and Stress Scale (DASS): The study of Validity and Reliability. Educational sciences : theory & practice 7, 1 (2007), 260.
[2]
American Psychiatric Association. 2013. Diagnostic and statistical manual of mental disorders : DSM-5. (5th ed. ed.). American Psychiatric Pub, Arlington, VA.
[3]
David H Barlow. 2001. Anxiety and its disorders: the nature and treatment of anxiety and panic. Guilford Press.
[4]
Megan Christina Barnsley. 2012. The social consequences of defensive physiological states.
[5]
Miguel Barreda-Ángeles, Sara Aleix-Guillaume, and Alexandre Pereda-Baños. 2020. Users’ psychophysiological, vocal, and self-reported responses to the apparent attitude of a virtual audience in stereoscopic 360°-video. Virtual reality : The journal of the Virtual Reality Society 24, 2(2020), 289–302.
[6]
Aaron T Beck and Emily A.P Haigh. 2014. Advances in Cognitive Theory and Therapy: The Generic Cognitive Model. Annual review of clinical psychology 10, 1 (2014), 1–24.
[7]
Tom Beckers and Michelle G Craske. 2017. Avoidance and decision making in anxiety: An introduction to the special issue. Behaviour research and therapy 96 (2017), 1–2.
[8]
Mehdi Boukhechba, Anna N Baglione, and Laura E Barnes. 2020. Leveraging Mobile Sensing and Machine Learning for Personalized Mental Health Care. Ergonomics in design 28, 4 (2020), 18–23.
[9]
Jason J Braithwaite, Derrick G Watson, Robert Jones, and Mickey Rowe. 2013. A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. Psychophysiology 49, 1 (2013), 1017–1034.
[10]
John A Chalmers, James A J Heathers, Maree J Abbott, Andrew H Kemp, and Daniel S Quintana. 2016. Worry is associated with robust reductions in heart rate variability: a transdiagnostic study of anxiety psychopathology. BMC Psychology 4, 1 (2016), 32–32.
[11]
John A Chalmers, Daniel S Quintana, Maree J-Anne Abbott, and Andrew H Kemp. 2014. Anxiety Disorders are Associated with Reduced Heart Rate Variability: A Meta-Analysis. Frontiers in psychiatry 5 (2014), 80–80.
[12]
Hsin-An Chang, Chuan-Chia Chang, Nian-Sheng Tzeng, Terry B J Kuo, Ru-Band Lu, and San-Yuan Huang. 2013. Generalized anxiety disorder, comorbid major depression and heart rate variability: a case-control study in taiwan. Psychiatry investigation 10, 4 (2013), 326–335.
[13]
Nele Dael, Marcello Mortillaro, and Klaus R Scherer. 2012. Emotion Expression in Body Action and Posture. Emotion (Washington, D.C.) 12, 5 (2012), 1085–1101.
[14]
Damien Dupré, Nicole Andelic, Daniel Stephen Moore, Gawain Morrison, and Gary John McKeown. 2020. Analysis of physiological changes related to emotions during a zipline activity. Sports engineering 23, 1 (2020).
[15]
T Ehring. 2014. Cognitive Theory. In The Wiley Handbook of Anxiety Disorders. John Wiley & Sons, Ltd, Chichester, UK, 104–124.
[16]
Mohamed Elgendi and Carlo Menon. 2019. Assessing Anxiety Disorders Using Wearable Devices: Challenges and Future Directions. Brain sciences 9, 3 (2019), 50.
[17]
A Ellis. 1991. The revised ABC’s of rational-emotive therapy (RET). Journal of rational-emotive and cognitive-behavior therapy 9, 3(1991), 139–172.
[18]
Norman S Endler and Nancy L Kocovski. 2001. State and trait anxiety revisited. Journal of anxiety disorders 15, 3 (2001), 231–245.
[19]
Thane M Erickson, Michelle G Newman, and Jamie L Tingey. 2020. Worry and rumination.(2020).
[20]
Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the 18th ACM international conference on multimedia(MM ’10). ACM, 1459–1462.
[21]
Rafael Christophe Freire, Rafael Ferreira-Garcia, Mariana Costa Cabo, Renan Machado Martins, and Antonio Egidio Nardi. 2020. Panic attack provocation in panic disorder patients with a computer simulation. Journal of affective disorders 264 (2020), 498–505.
[22]
Lior Galili, Ofer Amir, and Eva Gilboa-Schechtman. 2013. Acoustic properties of dominance and request utterances in social anxiety. Journal of social and clinical psychology 32, 6 (2013), 651–673.
[23]
Denis Gerstorf, Karen L Siedlecki, Elliot M Tucker-Drob, and Timothy A Salthouse. 2009. Within-person variability in state anxiety across adulthood: Magnitude and associations with between-person characteristics. International journal of behavioral development 33, 1(2009), 55–64.
[24]
Alexander M Goberman, Stephanie Hughes, and Todd Haydock. 2011. Acoustic characteristics of public speaking: Anxiety and practice effects. Speech communication 53, 6 (2011), 867–876.
[25]
Dan W Grupe and Jack B Nitschke. 2013. Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective. Nature reviews. Neuroscience 14, 7 (2013), 488–501.
[26]
Ritchie Hannah and Roser Max. 2018. Mental Health. Our World in Data (2018). https://ourworldindata.org/mental-health.
[27]
Trevor J Hastie and Robert J Tibshirani. 2017. Generalized additive models. Routledge.
[28]
RG Heimberg, KJ Horner, HR Juster, 1999. Psychometric properties of the Liebowitz Social Anxiety Scale. Psychological medicine 29, 1 (1999), 199–212.
[29]
Matthias Heyne, Donald Derrick, and Jalal Al-Tamimi. 2019. Native Language Influence on Brass Instrument Performance: An Application of Generalized Additive Mixed Models (GAMMs) to Midsagittal Ultrasound Images of the Tongue. Frontiers in psychology 10 (2019), 2597–2597.
[30]
Stefan G Hofmann, Stefan M Schulz, Sanna Heering, Frederick Muench, and Lynn F Bufka. 2010. Psychophysiological correlates of generalized anxiety disorder with or without comorbid depression. International journal of psychophysiology 78, 1 (2010), 35–41.
[31]
Emily M Johnson and Meredith E Coles. 2013. Failure and Delay in Treatment-Seeking Across Anxiety Disorders. Community mental health journal 49, 6 (2013), 668–674.
[32]
Ronald C Kessler, Maria Petukhova, Nancy A Sampson, Alan M Zaslavsky, and Hans-Ullrich Wittchen. 2012. Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. International journal of methods in psychiatric research 21, 3(2012), 169–184.
[33]
R Kher. 2019. Signal processing techniques for removing noise from ECG signals. Journal of Biomedical Engineering and Research 3 (2019), 1–9.
[34]
A Konnopka and H Koenig. 2020. Economic Burden of Anxiety Disorders: A Systematic Review and Meta-Analysis. PharmacoEconomics 38, 1 (2020), 25–37.
[35]
SD Kreibig. 2010. Autonomic nervous system activity in emotion: A review. Biological psychology 84, 3 (2010), 394–421.
[36]
Martin Lang, Jan Krátký, John H Shaver, Danijela Jerotijević, and Dimitris Xygalatas. 2015. Effects of Anxiety on Spontaneous Ritualized Behavior. Current biology 25, 14 (2015), 1892–1897.
[37]
Amy E Lawrence, Ronald C Kessler, Dan J Stein, 2008. Assessment of Anxiety Disorders. In Oxford Handbook of Anxiety and Related Disorders. Oxford Library of Psychology, Vol. 1. Oxford University Press.
[38]
DM Lipnicki and DG Byrne. 2008. An Effect of Posture on Anticipatory Anxiety. International journal of neuroscience 118, 2 (2008), 227–237.
[39]
Jeongok Logan, SeonAn Yeo, Suk-Sun Kim, and Mijung Lee. 2018. Anxiety and physical inactivity: breaking the vicious circle. Learning disability practice 21, 2 (2018), 15.
[40]
Valerie A MacIntyre, Peter D MacIntyre, and Geoff Carre. 2010. Heart Rate Variability as a Predictor of Speaking Anxiety. Communication research reports 27, 4 (2010), 286–297.
[41]
D Makowski, T Pham, Z.J Lau, J.C Brammer, F Lespinasse, H Pham, C Schölzel, and S.H.A Chen. 2021. NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior research methods(2021).
[42]
Iris Mauss, Frank Wilhelm, and James Gross. 2004. Is there less to social anxiety than meets the eye? Emotion experience, expression, and bodily responding. Cognition and emotion 18, 5 (2004), 631–642.
[43]
RE McCabe. 2015. Subjective units of distress scale. Phobias 18(2015), 361.
[44]
Alicia E Meuret, David Rosenfield, Frank H Wilhelm, Enlu Zhou, Ansgar Conrad, Thomas Ritz, and Walton T Roth. 2011. Do Unexpected Panic Attacks Occur Spontaneously?Biological psychiatry (1969) 70, 10 (2011), 985–991.
[45]
A Mkrtchian 2017. Modelling avoidance in mood and anxiety disorders using reinforcement-learning. Biological psychiatry (1969) 82, 7 (2017), 532–539.
[46]
Jennifer Lara Maria Mumm, Lena Pyrkosch, Jens Plag, Patrick Nagel, 2019. Heart rate variability in patients with agoraphobia with or without panic disorder remains stable during CBT but increases following in-vivo exposure. Journal of anxiety disorders 64 (2019), 16–23.
[47]
Mark Olfson, Mary Guardino, Elmer Struening, Franklin R Schneier, Fred Hellman, and Donald F Klein. 2000. Barriers to the Treatment of Social Anxiety. The American journal of psychiatry 157, 4 (2000), 521–527.
[48]
Turgut Özseven, Muharrem Düğenci, Ali Doruk, and Hilal I Kahraman. 2018. Voice traces of anxiety: acoustic parameters affected by anxiety disorder. Archives of Acoustics(2018), 625–636.
[49]
L Parkitny and J McAuley. 2010. The Depression Anxiety Stress Scale (DASS). Journal of physiotherapy 56, 3 (2010), 204.
[50]
H Posada-Quintero and K Chon. 2020. Innovations in electrodermal activity data collection and signal processing: A systematic review. Sensors 20, 2 (2020), 479.
[51]
Martin Reuter, Andrew J Cooper, Luke D Smillie, Sebastian Markett, and Christian Montag. 2015. A new measure for the revised reinforcement sensitivity theory: psychometric criteria and genetic validation. Frontiers in systems neuroscience 9 (2015), 38–38.
[52]
Jan Richter, Alfons O Hamm, Christiane A Pané-Farré, 2012. Dynamics of Defensive Reactivity in Patients with Panic Disorder and Agoraphobia: Implications for the Etiology of Panic Disorder. Biological psychiatry (1969) 72, 6 (2012), 512–520.
[53]
David Rosenfield, Enlu Zhou, Frank H Wilhelm, Ansgar Conrad, Walton T Roth, and Alicia E Meuret. 2010. Change point analysis for longitudinal physiological data: Detection of cardio-respiratory changes preceding panic attacks. Biological psychology 84, 1 (2010), 112–120.
[54]
Sainburg, T. 2020. noisereduce 1.1.0: Noise reduction in python using spectral gating. https://pypi.org/project/noisereduce/. Accessed: 2021-05-25.
[55]
Seeed Wiki. 2021. Grove - GSR Sensor. Seeed Technology Inc., Shenzhen, China.
[56]
Hashini Senaratne. 2019. Detecting Temporal Phases of Anxiety in The Wild: Toward Continuously Adaptive Self-Regulation Technologies. In 2019 International Conference on Multimodal Interaction (Suzhou, China) (ICMI ’19). Association for Computing Machinery, New York, NY, USA, 446–452.
[57]
Hashini Senaratne, Levin Kuhlmann, Kirsten Ellis, Glenn Melvin, and Sharon Oviatt. 2021. Anxiety Phases Dataset. https://doi.org/10.26180/15176082
[58]
F Shaffer and JP Ginsberg. 2017. An Overview of Heart Rate variability Metrics and Norms. Frontiers in public health 5 (2017), 258.
[59]
Vered Silber-Varod, Hamutal Kreiner, Ronen Lovett, Yossi Levi-Belz, and Noam Amir. 2016. Do social anxiety individuals hesitate more? The prosodic profile of hesitation disfluencies in Social Anxiety Disorder individuals. Proceedings of Speech Prosody 2016 31 (2016), 1211–1215.
[60]
M Sóskuthy. 2017. Generalised additive mixed models for dynamic analysis in linguistics: A practical introduction. arXiv preprint arXiv:1703.05339(2017).
[61]
LG Tereshchenko and ME Josephson. 2015. Frequency content and characteristics of ventricular conduction. Journal of electrocardiology 48, 6 (2015), 933–937.
[62]
JF Thayer. 2017. A Neurovisceral Integration Model of Heart Rate Variability. (2017).
[63]
Paul van Gent, Haneen Farah, Nicole van Nes, and Bart van Arem. 2019. HeartPy: A novel heart rate algorithm for the analysis of noisy signals. Transportation research. Part F, Traffic psychology and behaviour 66 (2019), 368–378.
[64]
KJ Wardenaar, CCW Lim, AO Al-Hamzawi, and J Alonso. 2018. The cross-national epidemiology of specific phobia in the World Mental Health Surveys - CORRIGENDUM. Psychological medicine 48, 5 (2018), 878–878.
[65]
Peter H Waxer. 1977. Nonverbal cues for anxiety: An examination of emotional leakage. Journal of abnormal psychology (1965) 86, 3 (1977), 306–314.
[66]
Justin W Weeks, Richard G Heimberg, and Reinhardt Heuer. 2011. Exploring the Role of Behavioral Submissiveness in Social Anxiety. Journal of social and clinical psychology 30, 3 (2011), 217–249.
[67]
Wanhui Wen, Guangyuan Liu, Zhi-Hong Mao, Wenjin Huang, Xu Zhang, Hui Hu, Jiemin Yang, and Wenyan Jia. 2020. Toward Constructing a Real-time Social Anxiety Evaluation System: Exploring Effective Heart Rate Features. IEEE transactions on affective computing 11, 1 (2020), 100–110.
[68]
WitMotion Team. 2020. User Manual WT901BLECL BLE5.0: Bluetooth 5.0 Inclinometer Sensor. WitMotion Shenzhen Co., Ltd., Shenzhen, China.
[69]
HU Wittchen, MB Stein, and RC Kessler. 1999. Social fears and social phobia in a community sample of adolescents and young adults: prevalence, risk factors and co-morbidity. Psychological medicine 29, 2 (1999), 309–323.
[70]
Simon N Wood. 2017. Generalized additive models: an introduction with R. CRC press.
[71]
World Health Organization. 2017. Depression and other common mental disorders: global health estimates. Technical Report.
[72]
Zephyr Technology. 2012. Zephyr™ BioHarness 3.0 User Manual. Zephyr Technology, Annapolis, MD, United States.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '21: Proceedings of the 2021 International Conference on Multimodal Interaction
October 2021
876 pages
ISBN:9781450384810
DOI:10.1145/3462244
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 October 2021

Permissions

Request permissions for this article.

Check for updates

Badges

  • Honorable Mention

Author Tags

  1. Anxiety
  2. Generalized additive mixed models
  3. Multimodal-multisensor dataset
  4. Objective detection
  5. Temporal phases

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICMI '21
Sponsor:
ICMI '21: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
October 18 - 22, 2021
QC, Montréal, Canada

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)72
  • Downloads (Last 6 weeks)13
Reflects downloads up to 01 Feb 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media