research-article

BreatheBuddy: Tracking Real-time Breathing Exercises for Automated Biofeedback Using Commodity Earbuds

Authors:

Md Mahbubur Rahman,

Mohsin Yusuf Ahmed,

Ebrahim Nemati,

Jun Alex GaoAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 6, Issue MHCI

Article No.: 213, Pages 1 - 18

https://doi.org/10.1145/3546748

Published: 20 September 2022 Publication History

Abstract

Breathing exercises reduce stress and improve overall mental well-being. There are various types of breathing exercises. Performing the exercises correctly may give the best outcome and doing it in wrong ways can sometimes have adverse effect. Providing real-time biofeedback can greatly improve the user experience in doing the right exercises in the right ways. In this paper, we present methods to passively track breathing biomarkers in real-time using wireless commodity earbuds and generate feedback on users' breathing performance. We use the earbud's low-power accelerometer to generate a comprehensive set of breathing biomarkers including breathing phase, breathing rate, depth of breathing, and breathing symmetry. We have conducted studies where the subjects performed different types of guided breathing exercises while wearing the earbuds. Our algorithms detect breathing phases with 90.91% F1-score and estimate breathing rate with 95.05% accuracy. We further show that our algorithms can be used to generate biofeedback towards designing engaging smartphone's user interactions that facilitate users to accurately perform various breathing exercises.

References

[1]

Asieh Ahani, Helane Wahbeh, Hooman Nezamfar, Meghan Miller, Deniz Erdogmus, and Barry Oken. 2014. Quantitative change of EEG and respiration signals during mindfulness meditation. Journal of neuroengineering and rehabilitation 11, 1 (2014), 1--11.

[2]

Ashfaaq Ahmed, R Gayatri Devi, and A Jothi Priya. 2021. Effect of Box Breathing Technique on Lung Function Test. (2021).

[3]

Beena Ahmed, Hira Mujeeb Khan, Jongyong Choi, and Ricardo Gutierrez-Osuna. 2015. ReBreathe: A calibration protocol that improves stress/relax classification by relabeling deep breathing relaxation exercises. IEEE Transactions on Affective Computing 7, 2 (2015), 150--161.

Digital Library

[4]

Farah Q AL-Khalidi, Reza Saatchi, Derek Burke, H Elphick, and Stephen Tan. 2011. Respiration Rate Monitoring Methods: A Review. Wiley Online Library on Pediatric Pulmonology (2011).

[5]

American Psychological Association. 2022. Stress in America. https://www.apa.org/news/press/releases/stress/.

[6]

Stephanie Balters, Matthew L Mauriello, So Yeon Park, James A Landay, and Pablo E Paredes. 2020. Calm commute: guided slow breathing for daily stress management in drivers. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--19.

Digital Library

[7]

Rummana Bari, Roy J Adams, Md Mahbubur Rahman, Megan Battles Parsons, Eugene H Buder, and Santosh Kumar. 2018. rConverse: Moment by Moment Conversation Detection Using a Mobile Respiration Sensor. ACM Interactive, Mobile, Wearable and u=Ubiquitous Technologies (IMWUT) (2018).

[8]

Rummana Bari, Md Mahbubur Rahman, Nazir Saleheen, Megan Battles Parsons, Eugene H Buder, and Santosh Kumar. 2020. Automated Detection of Stressful Conversations Using Wearable Physiological and Inertial Sensors. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 4, 4 (2020), 1--23.

Digital Library

[9]

Daniel M Bloomfield, Anthony Magnano, J Thomas Bigger Jr, Harold Rivadeneira, Michael Parides, and Richard C Steinman. 2001. Comparison of spontaneous vs. metronome-guided breathing on assessment of vagal modulation using RR variability. American Journal of Physiology-Heart and Circulatory Physiology 280, 3 (2001), H1145--H1150.

[10]

Kayla-Jade Butkow, Ting Dang, Andrea Ferlini, Dong Ma, and Cecilia Mascolo. 2021. Motion-resilient Heart Rate Monitoring with In-ear Microphones. arXiv preprint arXiv:2108.09393 (2021).

[11]

C3js. [n. d.]. A D3-based reusable chart library. https://c3js.org/.

[12]

Romit Roy Choudhury. 2021. Earable computing: A new area to think about. In Proceedings of the 22nd International Workshop on Mobile Computing Systems and Applications. 147--153.

Digital Library

[13]

Jérémy Frey, May Grabli, Ronit Slyper, and Jessica R Cauchard. 2018. Breeze: Sharing biofeedback through wearable technologies. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--12.

Digital Library

[14]

Vera C Goessl, Joshua E Curtiss, and Stefan G Hofmann. 2017. The effect of heart rate variability biofeedback training on stress and anxiety: a meta-analysis. Psychological medicine 47, 15 (2017), 2578--2586.

[15]

Paul Grossman, Frank H Wilhelm, and Martin Brutsche. 2010. Accuracy of ventilatory measurement employing ambulatory inductive plethysmography during tasks of everyday life. Biological psychology 84, 1 (2010), 121--128.

[16]

Tian Hao, Chongguang Bi, Guoliang Xing, Roxane Chan, and Linlin Tu. 2017. MindfulWatch: A smartwatch-based system for real-time respiration monitoring during meditation. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1--19.

Digital Library

[17]

Samsung Health. 2022. Samsung Health. https://play.google.com/store/apps/details?id=com.sec.android.app.shealth& hl=en_US&gl=US/.

[18]

Javier Hernandez, Yin Li, James M Rehg, and Rosalind W Picard. 2015. Cardiac and respiratory parameter estimation using head-mounted motion-sensitive sensors. EAI Endorsed Transactions on Pervasive Health and Technology (2015).

[19]

Javier Hernandez, Daniel McDuff, and Rosalind W Picard. 2015. Biowatch: estimation of heart and breathing rates from wrist motions. In 2015 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). IEEE, 169--176.

[20]

Hexoskin. 2022. Hexoskin, Vital Sign Monitoring Platform. https://www.hexoskin.com/pages/health-research/.

[21]

Karen Hovsepian, Mustafa Al'Absi, Emre Ertin, Thomas Kamarck, Motohiro Nakajima, and Santosh Kumar. 2015. cStress: Towards a Gold Standard for Continuous Stress Assessment in the Mobile Environment. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing.

Digital Library

[22]

Agni Kumar, Vikramjit Mitra, Carolyn Oliver, Adeeti Ullal, Matt Biddulph, and Irida Mance. 2021. Estimating Respiratory Rate From Breath Audio Obtained Through Wearable Microphones. arXiv preprint arXiv:2107.14028 (2021).

[23]

Jinmo Lee, Neska Elhaouij, and Rosalind Picard. 2021. AmbientBreath: Unobtrusive Just-in-time Breathing Intervention Using Multi-sensory Stimulation and its Evaluation in a Car Simulator. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--30.

Digital Library

[24]

Paul M Lehrer and Richard Gevirtz. 2014. Heart rate variability biofeedback: how and why does it work? Frontiers in psychology 5 (2014), 756.

[25]

Haipeng Liu, John Allen, Dingchang Zheng, and Fei Chen. 2019. Recent development of respiratory rate measurement technologies. Physiological measurement 40, 7 (aug 2019), 07TR01. https://doi.org/10.1088/1361--6579/ab299e

[26]

Alexis Martin and Jérémie Voix. 2017. In-ear audio wearable: Measurement of heart and breathing rates for health and safety monitoring. IEEE Transactions on Biomedical Engineering 65, 6 (2017), 1256--1263.

[27]

MonitoredRX. 2022. GoSpiro. http://monitoredrx.com/wp-content/uploads/2013/06/GoSpiro_IFU_2018.pdf/.

[28]

G Sunil Naik, GS Gaur, and GK Pal. 2018. Effect of modified slow breathing exercise on perceived stress and basal cardiovascular parameters. International journal of yoga 11, 1 (2018), 53.

[29]

Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaniel Watson. 2015. Contactless Sleep Apnea Detection on Smartphones. In ACM international conference on Mobile Systems, Applications, and Services (MobiSys).

[30]

Ebrahim Nemati, Shibo Zhang, Tousif Ahmed, Md Mahbubur Rahman, Jilong Kuang, and Alex Gao. 2021. CoughBuddy: Multi-Modal Cough Event Detection Using Earbuds Platform. In 2021 IEEE 17th International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE, 1--4.

[31]

Shahriar Nirjon, Robert F Dickerson, Qiang Li, Philip Asare, John A Stankovic, Dezhi Hong, Ben Zhang, Xiaofan Jiang, Guobin Shen, and Feng Zhao. 2012. Musicalheart: A hearty way of listening to music. In Proceedings of the 10th ACM Conference on Embedded Network Sensor Systems. 43--56

Digital Library

[32]

Pablo E Paredes, Yijun Zhou, Nur Al-Huda Hamdan, Stephanie Balters, Elizabeth Murnane, Wendy Ju, and James A Landay. 2018. Just breathe: In-car interventions for guided slow breathing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1--23.

Digital Library

[33]

Md Mahbubur Rahman, Tousif Ahmed, Mohsin Yusuf Ahmed, Ebrahim Nemati, Minh Dinh, Nathan Folkman, Md Mehedi Hasan, Jilong Kuang, and Jun Alex Gao. 2021. Towards Motion-Aware Passive Resting Respiratory Rate Monitoring Using Earbuds. In 2021 IEEE 17th International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE, 1--4.

[34]

Md Mahbubur Rahman, Rummana Bari, Amin Ahsan Ali, Moushumi Sharmin, Andrew Raij, Karen Hovsepian, Syed Monowar Hossain, Emre Ertin, Ashley Kennedy, David H Epstein, et al. 2014. Are We There Yet?: Feasibility of Continuous Stress Assessment via Wireless Physiological Sensors. In ACM Conference on Bioinformatics, Computational Biology, and Health Informatics.

Digital Library

[35]

Md Mahbubur Rahman, Viswam Nathan, Ebrahim Nemati, Korosh Vatanparvar, Mohsin Ahmed, and Jilong Kuang. 2019. Towards reliable data collection and annotation to extract pulmonary digital biomarkers using mobile sensors. In Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare. 179--188.

Digital Library

[36]

Md Mahbubur Rahman, Ebrahim Nemati, Viswam Nathan, and Jilong Kuang. 2018. InstantRR: Instantaneous Respiratory Rate Estimation on Context-aware Mobile Devices. In EAI International Conference on Body Area Networks.

[37]

Ruth Ravichandran, Elliot Saba, Ke-Yu Chen, Mayank Goel, Sidhant Gupta, and Shwetak N Patel. 2015. WiBreathe: Estimating Respiration Rate Using Wireless Signals in Natural Settings in the Home. In IEEE International Conference on Pervasive Computing and Communications (PerCom).

[38]

Tuomas Reinvuo, Manne Hannula, Hannu Sorvoja, Esko Alasaarela, and R Myllyla. 2006. Measurement of Respiratory Rate with High-resolution Accelerometer and EMFit Pressure Sensor. In IEEE Sensors Applications Symposium.

[39]

Yanzhi Ren, Chen Wang, Yingying Chen, and Jie Yang. 2014. Poster: Hearing Your Breathing: Fine-grained Sleep Monitoring Using Smartphones. In ACM International conference on Mobile Computing and Networking.

[40]

Yanzhi Ren, Chen Wang, Jie Yang, and Yingying Chen. 2015. Fine-grained Sleep Monitoring: Hearing Your Breathing with Smartphones. In IEEE Conference on Computer Communications (INFOCOM).

[41]

BCB Richard Gevirtz PhD. 2013. The promise of heart rate variability biofeedback: Evidence-based applications. Biofeedback (Online) 41, 3 (2013), 110.

[42]

Tobias Röddiger, Christopher Clarke, Daniel Wolffram, Matthias Budde, and Michael Beigl. 2021. EarRumble: Discreet Hands-and Eyes-Free Input by Voluntary Tensor Tympani Muscle Contraction. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--14.

Digital Library

[43]

Tobias Röddiger, Daniel Wolffram, David Laubenstein, Matthias Budde, and Michael Beigl. 2019. Towards respiration rate monitoring using an in-ear headphone inertial measurement unit. In Proceedings of the 1st International Workshop on Earable Computing. 48--53.

Digital Library

[44]

Matthew EB Russell, April B Scott, Ian A Boggero, and Charles R Carlson. 2017. Inclusion of a rest period in diaphragmatic breathing increases high frequency heart rate variability: Implications for behavioral therapy. Psychophysiology 54, 3 (2017), 358--365.

[45]

Maya C Schumer, Emily K Lindsay, and J David Creswell. 2018. Brief mindfulness training for negative affectivity: A systematic review and meta-analysis. Journal of Consulting and Clinical Psychology 86, 7 (2018), 569.

[46]

Chen-Hsuan Shih, Naofumi Tomita, Yanick X Lukic, Álvaro Hernández Reguera, Elgar Fleisch, and Tobias Kowatsch. 2019. Breeze: Smartphone-based Acoustic Real-time Detection of Breathing Phases for a Gamified Biofeedback Breathing Training. ACM Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (2019).

Digital Library

[47]

Martin Siepmann, Volkan Aykac, Jana Unterdörfer, Katja Petrowski, and Michael Mueck-Weymann. 2008. A pilot study on the effects of heart rate variability biofeedback in patients with depression and in healthy subjects. Applied psychophysiology and biofeedback 33, 4 (2008), 195--201.

[48]

Ranganatha Sitaram, Tomas Ros, Luke Stoeckel, Sven Haller, Frank Scharnowski, Jarrod Lewis-Peacock, Nikolaus Weiskopf, Maria Laura Blefari, Mohit Rana, Ethan Oblak, et al. 2017. Closed-loop Brain Training: the Science of Neurofeedback. Nature Reviews Neuroscience (2017).

[49]

Andrew Steptoe and Mika Kivimäki. 2012. Stress and cardiovascular disease. Nature Reviews Cardiology 9, 6 (2012), 360--370.

[50]

Xiao Sun, Li Qiu, Yibo Wu, Yeming Tang, and Guohong Cao. 2017. Sleepmonitor: Monitoring Respiratory Rate and Body Position During Sleep Using Smartwatch. ACM Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (2017).

Digital Library

[51]

Joachim Taelman, Steven Vandeput, Elke Vlemincx, Arthur Spaepen, and Sabine Van Huffel. 2011. Instantaneous changes in heart rate regulation due to mental load in simulated office work. European journal of applied physiology 111, 7 (2011), 1497--1505.

[52]

Peggy A Thoits. 2010. Stress and health: Major findings and policy implications. Journal of health and social behavior 51, 1_suppl (2010), S41--S53.

[53]

Ilse Van Diest, Karen Verstappen, André E Aubert, Devy Widjaja, Debora Vansteenwegen, and Elke Vlemincx. 2014. Inhalation/exhalation ratio modulates the effect of slow breathing on heart rate variability and relaxation. Applied psychophysiology and biofeedback 39, 3--4 (2014), 171--180.

[54]

Andrew Weil. 2017. Three breathing exercises. DrWeil. com (2017).

[55]

Xuhai Xu, Haitian Shi, Xin Yi, Wenjia Liu, Yukang Yan, Yuanchun Shi, Alex Mariakakis, Jennifer Mankoff, and Anind K Dey. 2020. Earbuddy: Enabling on-face interaction via wireless earbuds. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--14.

Digital Library

Cited By

Hu CKandappu TLiu YMascolo CMa D(2024)BreathPro: Monitoring Breathing Mode during Running with EarablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596078:2(1-25)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659607
Ahmed MRahman AWang ZRucker MBarnes LGanesan DLane NShi W(2024)A Resource Efficient System for On-Smartwatch Audio ProcessingProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3698866(1805-1807)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3698866
Canhoto JNeto PBarbosa TSantos JCampos IJunior GCoutinho JBrito MBrito ABrandão DAndrade ALeitão HCampos S(2024)Application of time series analysis to classify therapeutic breathing patternsSmart Health10.1016/j.smhl.2024.10046032(100460)Online publication date: Jun-2024
https://doi.org/10.1016/j.smhl.2024.100460
Show More Cited By

Index Terms

BreatheBuddy: Tracking Real-time Breathing Exercises for Automated Biofeedback Using Commodity Earbuds
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools

Recommendations

BreathePulse: Peripheral Guided Breathing via Implicit Airflow Cues for Information Work

Workplace stress contributes to poor performance and adverse health outcomes, yet current stress management tools often fall short in the fast-paced modern workforce. Guided slow breathing is a promising intervention for stress and anxiety, with ...
On the Production and Measurement of Cardiac Sounds in the Ear Canal
UbiComp '24: Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing

Over the past several years, a growing body of literature has proposed systems that use earable-based acoustic sensing to assess cardiac function. These works have offered various explanations of how in-ear cardiac audio is produced. Most claim that the ...
Manifesting Breath: Empirical Evidence for the Integration of Shape-changing Biofeedback-based Artefacts within Digital Mental Health Interventions
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Digital interventions are often used to support people with mental health conditions, but low engagement frequently reduces their effectiveness. We investigate the use of a Physical Artefact for Well-being Support (PAWS) to improve engagement and ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 6, Issue MHCI

MHCI

September 2022

852 pages

EISSN:2573-0142

DOI:10.1145/3564624

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 September 2022

Published in PACMHCI Volume 6, Issue MHCI

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
450
Total Downloads

Downloads (Last 12 months)164
Downloads (Last 6 weeks)16

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hu CKandappu TLiu YMascolo CMa D(2024)BreathPro: Monitoring Breathing Mode during Running with EarablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596078:2(1-25)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659607
Ahmed MRahman AWang ZRucker MBarnes LGanesan DLane NShi W(2024)A Resource Efficient System for On-Smartwatch Audio ProcessingProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3698866(1805-1807)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3698866
Canhoto JNeto PBarbosa TSantos JCampos IJunior GCoutinho JBrito MBrito ABrandão DAndrade ALeitão HCampos S(2024)Application of time series analysis to classify therapeutic breathing patternsSmart Health10.1016/j.smhl.2024.10046032(100460)Online publication date: Jun-2024
https://doi.org/10.1016/j.smhl.2024.100460
Karman ISun YSoroushmojdehi RSilva JMohsenvand M(2024)Generating Breathing Patterns in Real-Time: Low-Latency Respiratory Phase Tracking From 25 Hz PPGIoT Technologies and Wearables for HealthCare10.1007/978-3-031-71911-0_8(97-116)Online publication date: 23-Oct-2024
https://doi.org/10.1007/978-3-031-71911-0_8
Uygun MBae YChoi YPark D(2023)Development of Personalized Respiratory Training Device with Real-time Feedback for Respiratory Muscle StrengtheningPhysical Therapy Rehabilitation Science10.14474/ptrs.2023.12.3.25112:3(251-258)Online publication date: 30-Sep-2023
https://doi.org/10.14474/ptrs.2023.12.3.251
Rashid NRahman MAhmed TKuang JGao J(2023)BreathIE: Estimating Breathing Inhale Exhale Ratio Using Motion Sensor Data from Consumer EarbudsICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10096084(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10096084
Ahmed TRahman MNemati EKuang JGao A(2023)Mouth Breathing Detection Using Audio Captured Through EarbudsICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10095793(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10095793
Jin YRahman MAhmed TMukharesh LKuang JGao A(2023)VTMonitor: Tidal Volume Estimation Using Earbuds2023 IEEE 19th International Conference on Body Sensor Networks (BSN)10.1109/BSN58485.2023.10331516(1-5)Online publication date: 9-Oct-2023
https://doi.org/10.1109/BSN58485.2023.10331516

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents