ASAR Dataset and Computational Model for Affective State Recognition During ARAT Assessment for Upper Extremity Stroke Survivors
Pages 11 - 15
Abstract
Stroke is one of the leading global causes of disability and many stroke survivors must deal with challenging movement impairment daily. Stroke rehabilitation therapy is expensive, physically, and emotionally challenging, can be difficult to access, and is time-consuming for patients and clinicians alike. As a result, there is an opportunity to develop automated or semi-automated systems to assist clinicians in efficient and accurate stroke patient movement assessment. In this work, we present the ASAR (Affective State for ARAT Rehab) dataset where we use custom label classes (neutral, engagement, and pain) to annotate video data of stroke survivors performing the standardized Action Research Arm Test (ARAT) for upper extremity assessment. 106 patients participated in this study and each patient performed, or attempted to perform, each of the 19 ARAT assessment tasks. In our preliminary analysis, we annotated the frames of 126 videos for 10 patients. Using a multimodal model, we achieved an accuracy of 0.77 ± 0.06 in per-frame state recognition on the ASAR dataset. This affective information will potentially increase the interpretability of clinician’s scores of the stroke survivor’s ARAT performance and thus could assist in the design and implementation of semi-automated systems for stroke assessment and rehabilitation.
Supplementary Material
In this supplementary document, we have added a few figures that are required to explain the proposed system. (Supplementary Document.pdf)
- Download
- 186.54 KB
References
[1]
Tamim Ahmed, Kowshik Thopalli, Thanassis Rikakis, Pavan Turaga, Aisling Kelliher, Jia-Bin Huang, and Steven L Wolf. 2021. Automated movement assessment in stroke rehabilitation. Frontiers in Neurology 12 (2021), 720650.
[2]
Saad Albawi, Tareq Abed Mohammed, and Saad Al-Zawi. 2017. Understanding of a convolutional neural network. In 2017 international conference on engineering and technology (ICET). Ieee, 1–6.
[3]
Floor E Buma, Mathijs Raemaekers, Gert Kwakkel, and Nick F Ramsey. 2015. Brain function and upper limb outcome in stroke: a cross-sectional fMRI study. PLoS One 10, 10 (2015), e0139746.
[4]
Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7291–7299.
[5]
Aggelina Chatziagapi, Georgios Paraskevopoulos, Dimitris Sgouropoulos, Georgios Pantazopoulos, Malvina Nikandrou, Theodoros Giannakopoulos, Athanasios Katsamanis, Alexandros Potamianos, and Shrikanth Narayanan. 2019. Data Augmentation Using GANs for Speech Emotion Recognition. In Interspeech. 171–175.
[6]
James W Cooley, Peter AW Lewis, and Peter D Welch. 1969. The fast Fourier transform and its applications. IEEE Transactions on Education 12, 1 (1969), 27–34.
[7]
Abhinav Dhall, OV Ramana Murthy, Roland Goecke, Jyoti Joshi, and Tom Gedeon. 2015. Video and image based emotion recognition challenges in the wild: Emotiw 2015. In Proceedings of the 2015 ACM on international conference on multimodal interaction. 423–426.
[8]
Eva-Lotta Glader, Birgitta Stegmayr, and Kjell Asplund. 2002. Poststroke fatigue: a 2-year follow-up study of stroke patients in Sweden. Stroke 33, 5 (2002), 1327–1333.
[9]
Eduardo J Santos González and Kyla McMullen. 2020. The design of an algorithmic modal music platform for eliciting and detecting emotion. In 2020 8th international winter conference on brain-computer interface (bci). IEEE, 1–3.
[10]
Wei Han, Cheong-Fat Chan, Chiu-Sing Choy, and Kong-Pang Pun. 2006. An efficient MFCC extraction method in speech recognition. In 2006 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 4–pp.
[11]
Ashish B Ingale and DS Chaudhari. 2012. Speech emotion recognition. International Journal of Soft Computing and Engineering (IJSCE) 2, 1 (2012), 235–238.
[12]
Apostolos Kalatzis, Laura Stanley, and Vishnunarayan Girishan Prabhu. 2021. Affective state classification in virtual reality environments using electrocardiogram and respiration signals. In 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). IEEE, 160–167.
[13]
Christos D Katsis, Nikolaos S Katertsidis, and Dimitrios I Fotiadis. 2011. An integrated system based on physiological signals for the assessment of affective states in patients with anxiety disorders. Biomedical Signal Processing and Control 6, 3 (2011), 261–268.
[14]
AR Fugl Meyer, L Jaasko, I Leyman, S Olsson, and S Steglind. 1975. The post stroke hemiplegic patient. I. A method for evaluation of physical performance. Scandinavian Journal of Rehabilitation Medicine 7, 1 (1975), 13–31.
[15]
Seok-Jin Park, Hee-Beom Lee, and Gi-Woo Kim. 2023. Eardrum-inspired soft viscoelastic diaphragms for CNN-based speech recognition with audio visualization images. Scientific Reports 13, 1 (2023), 6414.
[16]
Rosalind W Picard. 2000. Affective computing. MIT press.
[17]
Jesús Joel Rivas, Felipe Orihuela-Espina, and Luis Enrique Sucar. 2019. Recognition of affective states in virtual rehabilitation using late fusion with Semi-Naive Bayesian classifier. In Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare. 308–313.
[18]
Philipp Werner, Daniel Lopez-Martinez, Steffen Walter, Ayoub Al-Hamadi, Sascha Gruss, and Rosalind W Picard. 2019. Automatic recognition methods supporting pain assessment: A survey. IEEE Transactions on Affective Computing 13, 1 (2019), 530–552.
[19]
Steven L Wolf, Pamela A Catlin, Michael Ellis, Audrey Link Archer, Bryn Morgan, and Aimee Piacentino. 2001. Assessing Wolf motor function test as outcome measure for research in patients after stroke. Stroke 32, 7 (2001), 1635–1639.
[20]
Min Xu, Ling-Yu Duan, Jianfei Cai, Liang-Tien Chia, Changsheng Xu, and Qi Tian. 2004. HMM-based audio keyword generation. In Pacific-Rim Conference on Multimedia. Springer, 566–574.
[21]
Nuray Yozbatiran, Lucy Der-Yeghiaian, and Steven C Cramer. 2008. A standardized approach to performing the action research arm test. Neurorehabilitation and neural repair 22, 1 (2008), 78–90.
[22]
Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. 2019. A review of recurrent neural networks: LSTM cells and network architectures. Neural computation 31, 7 (2019), 1235–1270.
[23]
Richard D Zorowitz, Randall J Smout, Julie A Gassaway, and Susan D Horn. 2005. Usage of pain medications during stroke rehabilitation: the Post-Stroke Rehabilitation Outcomes Project (PSROP). Topics in stroke rehabilitation 12, 4 (2005), 37–49.
Index Terms
- ASAR Dataset and Computational Model for Affective State Recognition During ARAT Assessment for Upper Extremity Stroke Survivors
Index terms have been assigned to the content through auto-classification.
Recommendations
Detecting Compensatory Trunk Movements in Stroke Survivors using a Wearable System
WearSys '17: Proceedings of the 2017 Workshop on Wearable Systems and ApplicationsBecause of the redundancy in the human body, stroke survivors often use atypical compensatory movements, such as reaching with trunk motion to compensate for deficits in arm function. Prior studies have established that minimizing these compensatory ...
Active Multimodal Stimulation in Rehabilitation of paretic upper limb after stroke: technical procedure
TEEM 2017: Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing MulticulturalityThe development of non-invasive transcranial stimulation techniques and their influence in Acquired Brain Injury (ABI) rehabilitation is increasing since nineties. There are several treatments for sensitive and motor upper limb recovery after a ...
Low-cost accelerometry-based posture monitoring system for stroke survivors
Assets '08: Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibilityThis paper reports a low-cost autonomous wearable accelerometry-based posture monitoring system for stroke survivors. The hardware part of the system consists of monitoring devices, each of which comprises of a three-axial accelerometer and a beeper, ...
Comments
Information & Contributors
Information
Published In
October 2023
434 pages
ISBN:9798400703218
DOI:10.1145/3610661
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 October 2023
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Funding Sources
- NIDILRR
- National Science Foundation
Conference
ICMI '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 263Total Downloads
- Downloads (Last 12 months)204
- Downloads (Last 6 weeks)16
Reflects downloads up to 03 Jan 2025
Other Metrics
Citations
Cited By
View allView Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in