A Modified POS-based rPPG with Real-time Deep Facial ROI Tracker and Pose Constrained Kalman Filter: M-POS rPPG with DFT and PCKF.
Authors: 
Yiming Liu, Rong Li, Min Liu, Binjie QinAuthors Info & Claims
Yiming Liu, Rong Li, Min Liu, Binjie QinAuthors Info & ClaimsPages 169 - 179
Abstract
Contactless measurement of heart rate(HR) based on videos can be essential for tele-monitoring medical system and public security. This paper proposed a modified POS-based remote photoplethysmography(rPPG) algorithm by replacing alpha tuning with singular value decomposition(SVD) to eliminate the specular reflection's spatiotemporal characteristic. What's more, we propose a novel landmark-based approach for a deep facial ROI tracker and face pose constrained Kalman filter to continuously and robustly track target facial ROIs for estimating HR from large head motion disturbances in rPPG. We demonstrate through experimental comparisons that the proposed method is more robust and accurate than the state-of-the-art rPPG-based methods in stable state. The mean absolute error (MAE) of HR estimation is 2.15 BPM lower than the POS and the AUC of HR estimation accuracy is 0.04 higher than POS in stable state. In motion state, the performance of our modified is a little bit better than POS.
References
[1]
World Health Organization. 2018. World health statistics 2018: monitoring health for the SDGs. Retrieved from http://www.indiaenvironmentportal.org.in/files/file/World%20Health%20Statistics%202018.pdf
[2]
Ambach, Wolfgang .2018. Detecting Concealed Information and Deception. Academic Press.
[3]
Chen, Xun. 2018. Video-based heart rate measurement: Recent advances and future prospects. IEEE Transactions on Instrumentation and Measurement, VOL. 68, NO. 10 (Oct. 2019), 3600-3615. DOI= https://dx.doi.org/10.1109/TIM.2018.2879706.
[4]
ZARETSKIY, A. P., 2019. Robust heart rate estimation using combined ECG and PPG signal processing. IOP Conference Series: Materials Science and Engineering. Vol. 537. No. 4. IOP Publishing. DOI= https://dx.doi.org/10.1088/1757-899X/537/4/042077
[5]
Kim Hye Geum, Cheon Eun Jin, Bai Dai Seg, Lee Young Hwan and Koo Bon Hoon. 2018. Stress and heart rate variability: A meta-analysis and review of the literature. Psychiatry investigation, 15(3), 235. DOI= https://dx.doi.org/10.30773/pi.2017.08.17
[6]
Li, S. Liu, L. Wu, J. Tang, B. and Li, D. 2018. Comparison and noise suppression of the transmitted and reflected photoplethysmography signals. BioMed research international, Vol.2018. (Sep. 2018), 9 pages. DOI= https://doi.org/10.1155/2018/4523593
[7]
Wang, W. den Brinker, A, C. Stuijk, S. 2016. Algorithmic principles of remote PPG. IEEE Transactions on Biomedical Engineering, Vol. No. 7. (Jul. 2017). 1479-1491.DOI= https://doi.org/10.1109/TBME.2016.2609282
[8]
Gasparini Francesca and Schettini Raimondo. 2006. Skin segmentation using multiple thresholding. Internet Imaging VII, volume 6061. 60610. International Society for Optics and Photonics. 16 January 2006, San Jose, California. 60610-60617 https://doi.org/10.1117/12.647446
[9]
Kazemi Vahid and Sullivan Josephine. 2014. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE conference on computer vision and pattern recognition. June 23-28, 2014. Columbus, OH, USA. 1867–1874. https://doi.org/10.1109/CVPR.2014.241
[10]
Lucas Bruce. D. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th International Joint Conference on Artificial Intelligence. 81(3):674-679.
[11]
Dan-Glauser Elise S and Gross James J. 2011. The temporal dynamics of two response-focused forms of emotion regulation: experiential, expressive, and autonomic consequences. Psychophysiology, 48(9), (Mar. 2011) 1309-1322. DOI= https://dx.doi.org/10.1111/j.1469-8986.2011. 01191.x
[12]
Galoogahi Hamed Kiani, Sim Terence and Lucey Simon. 2013. Multi-channel correlation filters. In Proceedings of the IEEE international conference on computer vision. March 03, 2013. Sydney, NSW, Australia. 3072–3079. https://doi.org/10.1109/ICCV.2013.381
[13]
Danelljan Martin, Bhat Goutam, Shahbaz Khan Fahad, 2017. Eco: Efficient convolution operators for tracking. Proceedings of the IEEE conference on computer vision and pattern recognition. July 21-26, 2017. Honolulu, HI, USA. 6931-6939. https://doi.org/10.1109/CVPR.2017.733
[14]
Guo Xiaojie, Li Siyuan, Zhang Jiawan, 2019. PFLD: A Practical Facial Landmark Detector. arXiv:1902.10859. Retrieved from http://arxiv.org/abs/1902.10859
[15]
Eom Ki Hwan, Lee Seung Joon, Kyung Yeo Sun, Lee Chang Won, Kim Min Chul, and Jung Kyung Kwon. 2011. Improved kalman filter method for measurement noise reduction in multi sensor rfid systems. Sensors, Vol.11 No.11. (Oct. 2011), 10266-10282. DOI= https://dx.doi.org/10.3390/s111110266
[16]
Song Pengfei, Manduca Armando., Trzasko, Joshua D., and Chen Shigao. 2016. Ultrasound small vessel imaging with block-wise adaptive local clutter filtering. IEEE transactions on medical imaging, Vol.36 No.1 (Sep. 2016), 251-262.DOI= https://doi.org/1010.1109/TMI.2016.2605819
[17]
Viola Paul and Jones Michael J. 2004. Robust real-time face detection. International journal of computer vision. ACM 57, 2, (May 2004), 137-154. DOI= https://doi.org/10.1023/B:VISI.0000013087.49260.fb
[18]
Sandler Mark, Howard Andrew, Zhu Menglong, Zhmoginov Andrey and Chen Liang Chieh. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. IEEE conference on computer vision and pattern recognition June 18-23, 2018. Salt Lake City, UT, USA. 4510-4520. https://doi.org/10.1109/CVPR.2018.00474
[19]
De Haan Gerard. and Jeanne Vincent. 2013. Robust pulse rate from chrominance-based rppg. IEEE Trans Biomed Eng. Vol.60 No.10.(Jun. 2013), 2878-2886. DOI= https://doi.org/10.1109/TBME.2013.2266196
[20]
Léon A.F. Ledoux, Brands Peter. J. and Hoeks Arnold. P. G. 1997. Reduction of the clutter component in Doppler ultrasound signals based on singular value decomposition: A simulation study. Ultrasonic imaging, Vol.19 No.1,(Jan. 1997) 1-18. DOI= https://doi.org/10.1177/016173469701900101
[21]
Niu Xuesong, Han Hu, Shan Shiguang and Chen Xilin. 2018, December. VIPL-HR: A multi-modal database for pulse estimation from less-constrained face video. Computer Vision. (May 2019), 562-576. Springer, Cham. DOI= https://dx.doi.org/10.1007/978-3-030-20873-8_36
[22]
Qi Huan, Guo Zhenyu, Chen Xun, Shen Zhiqi and Jane Wang, Z. 2017. Video-based human heart rate measurement using joint blind source separation. Biomedical Signal Processing and Control, (Jul. 2017) 31, 309-320. DOI= https://dx.doi.org/10.1016/j.bspc.2016.08.020
[23]
De Haan, Gerard., and Van Leest, A. 2014. Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiological measurement. Vol.35 No.9 (Aug. 2014), 1913-1926. DOI= https://dx.doi.org/10.1088/0967-3334/35/9/1913
[24]
Tran Quoc Viet, Su Shun Feng, Sun Wei and Tran Minh Quang. 2019. Adaptive Pulsatile Plane for Robust Noncontact Heart Rate Monitoring. IEEE Transactions on Systems. (Dec. 2019), page 13. DOI= https://doi.org/10.1109/TSMC.2019.2957159
[25]
Liu Yiming, Motion-Robust Multimodal Heart Rate Estimation Using BCG Fused Remote-PPG With Deep Facial ROI Tracker and Pose Constrained Kalman Filter. IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-15, 2021, Art no. 5007215.
Recommendations
Kalman filter-based facial emotional expression recognition
ACII'11: Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part IIn this work we examine the use of State-Space Models to model the temporal information of dynamic facial expressions. The later being represented by the 3D animation parameters which are recovered using 3D Candide model. The 3D animation parameters of ...
Pose-Robust Facial Expression Recognition Using View-Based 2D + 3D AAM
This paper proposes a pose-robust face tracking and facial expression recognition method using a view-based 2D 3D active appearance model (AAM) that extends the 2D 3D AAM to the view-based approach, where one independent face model is used for a ...
Pose-robust feature learning for facial expression recognition
Automatic facial expression recognition (FER) from non-frontal views is a challenging research topic which has recently started to attract the attention of the research community. Pose variations are difficult to tackle and many face analysis methods ...
Comments
Information & Contributors
Information
Published In
February 2021
194 pages
ISBN:9781450389112
DOI:10.1145/3456529
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 July 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- the Science and Technology Commission of Shanghai Municipality
Conference
ICCDA 2021
ICCDA 2021: 2021 The 5th International Conference on Compute and Data Analysis
February 2 - 4, 2021
Sanya, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 185Total Downloads
- Downloads (Last 12 months)24
- Downloads (Last 6 weeks)4
Reflects downloads up to 06 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format