Learning Continuous Mesh Representation with Spherical Implicit Surface
Pages 1 - 8
Abstract
As the most common representation for 3D shapes, mesh is often stored discretely with arrays of vertices and faces. However, 3D shapes in the real world are presented continuously. In this paper, we propose to learn a continuous representation for meshes with fixed topology, a common and practical setting in many faces-, hand-, and body-related applications. First, we split the template into multiple closed manifold genus-0 meshes so that each genus-0 mesh can be parameterized onto the unit sphere. Then we learn spherical implicit surface (SIS), which takes a spherical coordinate and a global feature or a set of local features around the coordinate as inputs, predicting the vertex corresponding to the coordinate as an output. Since the spherical coordinates are continuous, SIS can depict a mesh in an arbitrary resolution. SIS representation builds a bridge between discrete and continuous representation in 3D shapes. Specifically, we train SIS networks in a self-supervised manner for two tasks: a reconstruction task and a super-resolution task. Experiments show that our SIS representation is comparable with state-of-the-art methods that are specifically designed for meshes with a fixed resolution and significantly outperforms methods that work in arbitrary resolutions.
References
[1]
P. Achlioptas, O. Diamanti, I. Mitliagkas, and L. Guibas. Learning representations and generative models for 3d point clouds. In ICML, pages 40–49. PMLR, 2018.
[2]
M. Attene. A lightweight approach to repairing digitized polygon meshes. The Visual Computer, 26(11):1393–1406, Nov2010.
[3]
A. Baden, K. Crane, and M. Kazhdan. Mobius Registration. Computer Graphics Forum, 2018.
[4]
F. Bogo, J. Romero, G. Pons-Moll, and M. J. Black. Dynamic FAUST: Registering human bodies in motion. In CVPR, July2017.
[5]
C. Cao, Q. Hou, and K. Zhou. Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph., 33(4):43:1–43:10, July2014.
[6]
Y. Chen, S. Liu, and X. Wang. Learning continuous image representation with local implicit image function. arXiv preprint arXiv:, 2020.
[7]
G. P. T. Choi, Y. Leung-Liu, X. Gu, and L. M. Lui. Parallelizable global conformal parameterization of simply-connected surfaces via partial welding. SIAM Journal on Imaging Sciences, 13(3):1049–1083, 2020.
[8]
C. B. Choy, D. Xu, J. Gwak, K. Chen, and S. Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In B. Leibe, J. Matas, N. Sebe, and M. Welling, editors, ECCV, pages 628–644, Cham, 2016. Springer International Publishing.
[9]
H. Fan, H. Su, and L. J. Guibas. A point set generation network for 3d object reconstruction from a single image. In CVPR, July2017.
[10]
Z. Gao, J. Yan, G. Zhai, J. Zhang, Y. Yang, and X. Yang. Learning local neighboring structure for robust 3d shape representation. In AAAI, 2021.
[11]
Z. Gao, J. Zhang, Y. Guo, C. Ma, G. Zhai, and X. Yang. Semi-supervised 3d face representation learning from unconstrained photo collections. In CVPR Workshops, 2020.
[12]
K. Genova, F. Cole, A. Maschinot, A. Sarna, D. Vlasic, and W. T. Freeman. Unsupervised training for 3D morphable model regression. In CVPR, June2018.
[13]
R. Girdhar, D. F. Fouhey, M. Rodriguez, and A. Gupta. Learning a predictable and generative vector representation for objects. In B. Leibe, J. Matas, N. Sebe, and M. Welling, editors, ECCV, pages 484–499, 2016. Cham: Springer International Publishing.
[14]
C. Gotsman, X. Gu, and A. Sheffer. Fundamentals of spherical parameterization for 3d meshes. ACM Trans. Graph., 22(3):358–363, July2003.
[15]
T. Groueix, M. Fisher, V. G. Kim, B. C. Russell, and M. Aubry. 3D-CODED: 3D correspondences by deep deformation. In ECCV, September2018.
[16]
K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989.
[17]
A. Kanazawa, S. Tulsiani, A. A. Efros, and J. Malik. Learning category-specific mesh reconstruction from image collections. In ECCV, September2018.
[18]
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:, 2014.
[19]
D. Maturana and S. Scherer. Voxnet: A 3d convolutional neural network for real-time object recognition. In IROS, pages 922–928, 2015.
[20]
L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy networks: Learning 3d reconstruction in function space. In CVPR, June2019.
[21]
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor-thi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, editors, ECCV, pages 405–421, 2020. Cham: Springer International Publishing.
[22]
T. Müller, A. Evans, C. Schied, and A. Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4):102:1–102:15, July2022.
[23]
E. Ng, S. Ginosar, T. Darrell, and H. Joo. Body2hands: Learning to infer 3d hands from conversational gesture body dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11865–11874, 2021.
[24]
J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, June2019.
[25]
S. Peng, M. Niemeyer, L. Mescheder, M. Pollefeys, and A. Geiger. Convolutional occupancy networks. In A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, editors, ECCV, pages 523–540, 2020. Cham: Springer International Publishing.
[26]
C. R. Qi, H. Su, M. Niessner, A. Dai, M. Yan, and L. J. Guibas. Volumetric and multi-view CNNs for object classification on 3d data. In CVPR, June2016.
[27]
A. Ranjan, T. Bolkart, S. Sanyal, and M. J. Black. Generating 3D faces using convolutional mesh autoencoders. In ECCV, September2018.
[28]
M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng. Fourier features let networks learn high frequency functions in low dimensional domains. NeurIPS, 2020.
[29]
L. Tran and X. Liu. Nonlinear 3D face morphable model. In CVPR, June2018.
[30]
N. Verma, E. Boyer, and J. Verbeek. FeaStNet: Feature-steered graph convolutions for 3D shape analysis. In CVPR, June2018.
Recommendations
Smooth surface and triangular mesh: comparison of the area, the normals and the unfolding
SMA '02: Proceedings of the seventh ACM symposium on Solid modeling and applicationsReplacing a smooth surface with a triangular mesh (i.e., a polyedron) "close to it" leads to some errors. The geometric properties of the triangular mesh can be very different from the geometric properties of the smooth surface, even if both surfaces ...
Surface mesh regeneration considering curvatures
This work describes an automatic algorithm for unstructured mesh regeneration on arbitrarily shaped three-dimensional surfaces. The arbitrary surface may be: a triangulated mesh, a set of points, or an analytical surface (such as a collection of NURBS ...
Direct Spherical Parameterization Based on Surface Curvature
DMDCM '11: Proceedings of the 2011 Workshop on Digital Media and Digital Content ManagementIn this paper we propose a novel spherical parameterization for closed, genus-0, two-manifold, 3Dtriangular meshes. The key point of our method concerns the Gaussian curvature criterion involved, which makes it possible to detect iteratively salient ...
Comments
Information & Contributors
Information
Published In
Jan 2023
540 pages
Copyright © 2023.
Publisher
IEEE Press
Publication History
Published: 05 January 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Jan 2025