3D Human Body Shape and Pose Estimation from Depth Image
Authors: 
Lei Liu, Kangkan Wang, Jian YangAuthors Info & Claims
Lei Liu, Kangkan Wang, Jian YangAuthors Info & ClaimsPages 410 - 421
Abstract
This work addresses the problem of 3D human body shape and pose estimation from a single depth image. Most 3D human pose estimation methods based on deep learning utilize RGB images instead of depth images. Traditional optimization-based methods using depth images aim to establish point correspondences between the depth images and the template model. In this paper, we propose a novel method to estimate the 3D pose and shape of a human body from depth images. Specifically, based on the joints features and original depth features, we propose a spatial attention feature extractor to capture spatial local features of depth images and 3D joints by learning dynamic weights of the features. In addition, we generalize our method to real depth data through a weakly-supervised method. We conduct extensive experiments on SURREAL, Human3.6M, DFAUST, and real depth images of human bodies. The experimental results demonstrate that our 3D human pose estimation method can yield good performance.
References
[1]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
[2]
Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J.: End-to-end recovery of human shape and pose. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7122–7131, June 2018
[3]
Varol, G., et al.: Learning from synthetic humans. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 109–117, July 2017
[4]
Ionescu C, Papava D, Olaru V, and Sminchisescu C Human3.6M: large scale datasets and predictive methods for 3D human sensing in natural environments IEEE Trans. Pattern Anal. Mach. Intell. 2014 36 7 1325-1339
[5]
Bogo, F., Romero, J., Pons-Moll, G., Black, M.J.: Dynamic faust: registering human bodies in motion. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6233–6242, July 2017
[6]
Loper M, Mahmood N, Romero J, Pons-Moll G, and Black MJ SMPL: a skinned multi-person linear model ACM Trans. Graph. 2015 34 1-16
[7]
Bogo F, Kanazawa A, Lassner C, Gehler P, Romero J, and Black MJ Leibe B, Matas J, Sebe N, and Welling M Keep it SMPL: automatic estimation of 3D human pose and shape from a single image Computer Vision – ECCV 2016 2016 Cham Springer 561-578
[8]
Huang, Y., et al.: Towards accurate marker-less human shape and pose estimation over time. In: 2017 International Conference on 3D Vision (3DV), pp. 421–430. IEEE (2017)
[9]
Kolotouros, N., Pavlakos, G., Daniilidis, K.: Convolutional mesh regression for single-image human shape reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4501–4510 (2019)
[10]
Wei, L., Huang, Q., Ceylan, D., Vouga, E., Li, H.: Dense human body correspondences using convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1544–1553 (2016)
[11]
Kadkhodamohammadi, A., Gangi, A., de Mathelin, M., Padoy, N.: A multi-view RGB-D approach for human pose estimation in operating rooms. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 363–372. IEEE (2017)
[12]
Pavlakos, G., Zhou, X., Daniilidis, K.: Ordinal depth supervision for 3D human pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7307–7316 (2018)
[13]
Li, R., Cai, C., Georgakis, G., Karanam, S., Chen, T., Wu, Z.: Towards robust RGB-D human mesh recovery. arXiv preprint arXiv:1911.07383 (2019)
[14]
Kato, H., Ushiku, Y., Harada, T.: Neural 3D mesh renderer. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3907–3916 (2018)
[15]
Sun X, Xiao B, Wei F, Liang S, and Wei Y Ferrari V, Hebert M, Sminchisescu C, and Weiss Y Integral human pose regression Computer Vision – ECCV 2018 2018 Cham Springer 536-553
[16]
Loper M, Mahmood N, and Black MJ MoSh: motion and shape capture from sparse markers ACM Trans. Graph. (TOG) 2014 33 6 1-13
[17]
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
[18]
Yu, R., Saito, S., Li, H., Ceylan, D., Li, H.: Learning dense facial correspondences in unconstrained images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4723–4732 (2017)
[19]
Kolotouros, N., Pavlakos, G., Black, M.J., Daniilidis, K.: Learning to reconstruct 3D human pose and shape via model-fitting in the loop. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2252–2261 (2019)
[20]
Zhu, H., Zuo, X., Wang, S., Cao, X., Yang, R.: Detailed human shape estimation from a single image by hierarchical mesh deformation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4491–4500 (2019)
[21]
Guo, K., Xu, F., Wang, Y., Liu, Y., Dai, Q.: Robust non-rigid motion tracking and surface reconstruction using L0 regularization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3083–3091 (2015)
[22]
Vlasic, D., Baran, I., Matusik, W., Popović, J.: Articulated mesh animation from multi-view silhouettes. In: ACM SIGGRAPH 2008 papers, pp. 1–9 (2008)
[23]
Zhang, C., Pujades, S., Black, M.J., Pons-Moll, G.: Detailed, accurate, human shape estimation from clothed 3D scan sequences. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4191–4200
Recommendations
3D human pose and shape estimation with dense correspondence from a single depth image
AbstractWe propose a novel approach to estimate the 3D pose and shape of human bodies with dense correspondence from a single depth image. In contrast to most current 3D body model recovery methods from depth images that employ motion information of depth ...
Estimating 3d human body pose from stereo image sequences
GW'05: Proceedings of the 6th international conference on Gesture in Human-Computer Interaction and SimulationThis paper presents a novel method for estimating 3D human body pose from stereo image sequences based on top-down learning. Human body pose is represented by a linear combination of prototypes of 2D depth images and their corresponding 3D body models ...
Comments
Information & Contributors
Information
Published In
Oct 2020
788 pages
ISBN:978-3-030-60632-9
DOI:10.1007/978-3-030-60633-6
© Springer Nature Switzerland AG 2020.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 16 October 2020
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 02 Feb 2025