research-article

Region of Interest-Based 3D Inpainting of Cultural Heritage Artifacts

Authors:

Uma MudenagudiAuthors Info & Claims

Journal on Computing and Cultural Heritage (JOCCH), Volume 11, Issue 2

Article No.: 9, Pages 1 - 21

https://doi.org/10.1145/3131778

Published: 22 May 2018 Publication History

Abstract

In this article, we address the problem of 3D inpainting using an exemplar-based method for point clouds. 3D inpainting is a process of filling holes or missing regions in the reconstructed 3D models. Typically, inpainting methods addressed in the literature fill missing regions due to occlusions or inaccurate scanning of 3D models. However, we focus on scenarios involving naturally existing damaged models, which are partly broken or incomplete in the artifacts at cultural heritage sites. We propose an exemplar-based inpainting technique using the region of interest (ROI)-based method to inpaint the missing regions of the damaged model. The ROI of a 3D model is represented as a set of Riemannian manifolds, and metric tensor and Christoffel symbols are used as geometric features to capture the inherent geometry. We then decompose the ROI into basic shape regions, namely, spherical, conical, and cylindrical components, and identify the best-fit match for inpainting. Instead of using a single similar exemplar for inpainting, we select the most relevant best-fit region to fill the missing region from the basic shape regions library obtained from n similar exemplars. We demonstrate the performance of the proposed inpainting method on artifacts at UNESCO World Heritage site Hampi temples, India with varying complexities and sizes for both synthetically generated holes and real missing regions in 3D objects.

References

[1]

B. Amberg, S. Romdhani, and T. Vetter. 2007. Optimal step nonrigid ICP algorithms for surface registration. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 1--8.

[2]

Oscar Argudo, Pere Brunet, Antoni Chica, and Àlvar Vinacua. 2015. Biharmonic fields and mesh completion. Graph. Models 82 (2015), 137--148.

Digital Library

[3]

Marco Attene, Marcel Campen, and Leif Kobbelt. 2013. Polygon mesh repairing: An application perspective. ACM Comput. Surv. 45, 2, Article 15 (2013), 33 pages.

Digital Library

[4]

Loïc Barthe and Leif Kobbelt. 2004. Subdivision scheme tuning around extraordinary vertices. Comput. Aided Geom. Des. 21, 6 (2004), 561--583.

Digital Library

[5]

G. H. Bendels, R. Schnabel, and R. Klein. 2005. Detail-preserving surface inpainting. In Proceedings of the International Conference on Virtual Reality, Archaeology and Intelligent Cultural Heritage. Eurographics Association, 41--48.

Digital Library

[6]

Matthew Berger, Andrea Tagliasacchi, Lee Seversky, Pierre Alliez, Joshua Levine, Andrei Sharf, and Claudio Silva. 2014. State of the art in surface reconstruction from point clouds. In Eurographics - State of the Art Reports (EUROGRAPHICS star report), Vol. 1. 161--185.

[7]

Stephan Bischoff, Darko Pavic, and Leif Kobbelt. 2005. Automatic restoration of polygon models. ACM Trans. Graph. 24, 4 (2005), 1332--1352.

Digital Library

[8]

S. Boyé, G. Guennebaud, and C. Schlick. 2010. Least squares subdivision surfaces. Computer Graphics Forum 29, 7 (2010), 2021--2028.

[9]

V. Caselles, G. Haro, G. Sapiro, and J. Verdera. 2008. On geometric variational models for inpainting surface holes. Comput. Vis. Image Underst. 111, 3 (2008), 351--373.

Digital Library

[10]

Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 3 (2011), 27:1--27:27.

Digital Library

[11]

U. Clarenz, U. Diewald, G. Dziuk, M. Rumpf, and R. Rusu. 2004. A finite element method for surface restoration with smooth boundary conditions. Comput. Aided Geom. Des. 21, 5 (2004), 427--445.

Digital Library

[12]

Massimiliano Corsini, Paolo Cignoni, and Roberto Scopigno. 2012. Efficient and flexible sampling with blue noise properties of triangular meshes. IEEE Trans. Visual. Comput. Graphics 18, 6 (2012), 914--924.

Digital Library

[13]

Christopher M. Cyr and Benjamin B. Kimia. 2001. 3D object recognition using shape similarity-based aspect graph. In Proceedings of the International Conference on Computer Vision. 254--261.

[14]

James Davis, Stephen R. Marschner, Matt Garr, and Marc Levoy. 2002. Filling holes in complex surfaces using volumetric diffusion. In Proceedings of the International Symposium on 3D Data Processing, Visualization and Transmission. IEEE, 428--441.

[15]

Andrea Frome, Daniel Huber, Ravi Kolluri, Thomas Bulow, and Jitendra Malik. 2004. Recognizing objects in range data using regional point descriptors. In Proceedings of the European Conference on Computer Vision (ECCV).

[16]

Ran Gal, Ariel Shamir, Tal Hassner, Mark Pauly, and Daniel Cohen-Or. 2007. Surface reconstruction using local shape priors. In Proceedings of the Eurographics Symposium on Geometry Processing (SGP). Eurographics Association, 253--262.

Digital Library

[17]

Syed Altaf Ganihar, Shreyas Joshi, Shankar Setty, and Uma Mudenagudi. 2014. Metric tensor and Christoffel symbols based 3D object categorization. In Computer Vision-ACCV Workshops. Springer, 138--151.

Digital Library

[18]

Syed Altaf Ganihar, Shreyas Joshi, Shankar Shetty, and Uma Mudenagudi. 2014. Metric tensor and Christoffel symbols based 3D object categorization. In ACM SIGGRAPH Posters. 38:1--38:1.

Digital Library

[19]

Aleksey Golovinskiy, Vladimir G. Kim, and Thomas A. Funkhouser. 2009. Shape-based recognition of 3D point clouds in urban environments. In Proceedings of the International Conference on Computer Vision. IEEE, 2154--2161.

[20]

Xiaoyuan Guo, Jun Xiao, and Ying Wang. 2016. A survey on algorithms of hole filling in 3D surface reconstruction. The Visual Computer (2016), 1--11.

Digital Library

[21]

Yulan Guo, Mohammed Bennamoun, Ferdous Sohel, Min Lu, Jianwei Wan, and Ngai Ming Kwok. 2016. A comprehensive performance evaluation of 3D local feature descriptors. Int. J. Comput. Vision 116, 1 (2016), 66--89.

Digital Library

[22]

Gur Harary, Ayellet Tal, and Eitan Grinspun. 2014. Context-based coherent surface completion. ACM Trans. Graph. 33, 1 (2014), 5:1--5:12.

Digital Library

[23]

Gur Harary, Ayellet Tal, and Eitan Grinspun. 2014. Feature-preserving surface completion using four points. Comput. Graph. Forum 33, 5 (2014), 45--54.

Digital Library

[24]

C. Häne, C. Zach, A. Cohen, R. Angst, and M. Pollefeys. 2013. Joint 3D scene reconstruction and class segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 97--104.

Digital Library

[25]

Andrew Johnson and Martial Hebert. 1999. Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21, 5 (1999), 433--449.

Digital Library

[26]

J. Jost. 2011. Riemannian geometry and geometric analysis. In Springer Universitat texts (6 ed.). Springer-Verlag Berlin.

[27]

Tao Ju. 2004. Robust repair of polygonal models. ACM Trans. Graph. 23, 3 (2004), 888--895.

Digital Library

[28]

Tao Ju. 2009. Fixing geometric errors on polygonal models: A survey. J. Comput. Sci. Technol. 24, 1 (2009), 19--29.

Digital Library

[29]

Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. 2006. Poisson surface reconstruction. In Proceedings of the Eurographics Symposium on Geometry Processing (SGP). Eurographics Association, 61--70.

Digital Library

[30]

Vladimir G. Kim, Siddhartha Chaudhuri, Leonidas Guibas, and Thomas Funkhouser. 2014. Shape2Pose: Human-centric shape analysis. ACM Trans. Graph. 33, 4, Article 120 (2014), 12 pages.

Digital Library

[31]

Vladimir G. Kim, Wilmot Li, Niloy J. Mitra, Siddhartha Chaudhuri, Stephen DiVerdi, and Thomas Funkhouser. 2013. Learning part-based templates from large collections of 3D shapes. ACM Trans. Graph. 32, 4, Article 70 (2013), 12 pages.

Digital Library

[32]

Jan Knopp, Mukta Prasad, Geert Willems, Radu Timofte, and Luc Van Gool. 2010. Hough transform and 3D SURF for robust three dimensional classification. In Proceedings of the 11th ECCV: Part VI. Springer-Verlag.

Digital Library

[33]

Vladislav Kraevoy and Alla Sheffer. 2005. Template-based mesh completion. In Proceedings of the Eurographics Symposium on Geometry Processing (SGP). Eurographics Association.

Digital Library

[34]

Peter Liepa. 2003. Filling holes in meshes. In Proceedings of the Eurographics Symposium on Geometry Processing (SGP). Eurographics Association, 200--205.

Digital Library

[35]

Roberto Javier López-Sastre, A. García-Fuertes, Carolina Redondo-Cabrera, Francisco Javier Acevedo-Rodríguez, and Saturnino Maldonado-Bascón. 2013. Evaluating 3D spatial pyramids for classifying 3D shapes. Comput. Graphics 37, 5 (2013), 473--483.

Digital Library

[36]

Marianna Madry, Carl Henrik Ek, Renaud Detry, Kaiyu Hang, and Danica Kragic. 2012. Improving generalization for 3D object categorization with global structure histograms. In Proceedings of the International Conference on Intelligent Robots and Systems. IEEE, 1379--1386.

[37]

Christian Atanas Mueller. 2012. 3D Object Shape Categorization in Domestic Environments. Technical Report. Bonn-Rhein-Sieg University.

[38]

D. T. Nguyen, B. S. Hua, M. K. Tran, Q. H. Pham, and S. K. Yeung. 2016. A field model for repairing 3D shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5676--5684.

[39]

Constantine Papageorgiou and Tomaso Poggio. 2000. A trainable system for object detection. Int. J. Comput. Vision 38, 1 (2000), 15--33.

Digital Library

[40]

Mark Pauly, Niloy J. Mitra, Joachim Giesen, Markus H. Gross, and Leonidas J. Guibas. 2005. Example-based 3D scan completion. In Proceedings of the Eurographics Symposium on Geometry Processing (SGP). Eurographics Association, 23--32.

Digital Library

[41]

Jean-Philippe Pernot, George Moraru, and Philippe Véron. 2006. Filling holes in meshes using a mechanical model to simulate the curvature variation minimization. Comput. Graph. 30, 6 (2006), 892--902.

Digital Library

[42]

Axel Pinz. 2005. Object categorization. Found. Trends. Comput. Graph. Vis. 1, 4 (2005), 255--353.

Digital Library

[43]

Joshua Podolak and Szymon Rusinkiewicz. 2005. Atomic volumes for mesh completion. In Proceedings of the Eurographics Symposium on Geometry Processing (SGP). Eurographics Association.

Digital Library

[44]

Yann Quinsat and Claire Lartigue. 2015. Filling holes in digitized point cloud using a morphing-based approach to preserve volume characteristics. Int. J. Adv. Manuf. Technol. 81, 1 (2015), 411--421.

[45]

S. Rusinkiewicz and M. Levoy. 2001. Efficient variants of the ICP algorithm. In 3-D Digital Imaging and Modeling. 145--152.

[46]

Radu Bogdan Rusu, Nico Blodow, and Michael Beetz. 2009. Fast point feature histograms (FPFH) for 3D registration. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA’09). IEEE Press, 1848--1853.

Digital Library

[47]

Radu Bogdan Rusu, Gary R. Bradski, Romain Thibaux, and John Hsu. 2010. Fast 3D recognition and pose using the viewpoint feature histogram. In Proceedings of the International Conference on Intelligent Robots and Systems. IEEE, 2155--2162.

[48]

Pratyush Sahay and A. Rajagopalan. 2015. Geometric inpainting of 3D structures. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1--7.

[49]

Pratyush Sahay and Ambasamudram Narayanan Rajagopalan. 2012. Harnessing self-similarity for reconstruction of large missing regions in 3d models. In Proceedings of the International Conference on Pattern Recognition (ICPR). IEEE, 101--104.

[50]

Silvio Savarese and Fei-Fei Li. 2007. 3D generic object categorization, localization and pose estimation. In Proceedings of the International Conference on Computer Vision. IEEE, 1--8.

[51]

Silvio Savarese and Fei-Fei Li. 2008. View synthesis for recognizing unseen poses of object classes. In European Conference on Computer Vision (3) (Lecture Notes in Computer Science), Vol. 5304. Springer, 602--615.

Digital Library

[52]

Silvio Savarese and Fei-Fei Li. 2010. Multi-view object categorization and pose estimation. In Computer Vision: Detection, Recognition and Reconstruction. Vol. 285. Springer, 205--231.

[53]

R. Schnabel, R. Wahl, and R. Klein. 2007. Efficient RANSAC for point-cloud shape detection. Comput. Graphics Forum 26, 2 (2007), 214--226.

[54]

Henry Schneiderman. 2000. A Statistical Approach to 3D Object Detection Applied to Faces and Cars. Ph.D. Dissertation. Robotics Institute, Carnegie Mellon University.

Digital Library

[55]

S. Setty, S. A. Ganihar, and U. Mudenagudi. 2015. Framework for 3D object hole filling. In Proceedings of the 5th National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG). 1--4.

[56]

Andrei Sharf, Marc Alexa, and Daniel Cohen-Or. 2004. Context-based surface completion. ACM Trans. Graphics (TOG) 23, 3 (2004), 878--887.

Digital Library

[57]

Oana Sidi, Oliver van Kaick, Yanir Kleiman, Hao Zhang, and Daniel Cohen-Or. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM Trans. Graph. 30, 6, Article 126 (2011), 10 pages.

Digital Library

[58]

Shuran Song and Jianxiong Xiao. 2014. Sliding shapes for 3D object detection in depth images. In Proceedings of the European Conference on Computer Vision. Springer International Publishing, 634--651.

[59]

Robert W. Sumner and Jovan Popović. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3 (2004), 399--405.

Digital Library

[60]

Roberto Toldo, Umberto Castellani, and Andrea Fusiello. 2010. The bag of words approach for retrieval and categorization of 3D objects. Vis. Comput. 26, 10 (2010), 1257--1268.

Digital Library

[61]

Julien P. C. Valentin, Sunando Sengupta, Jonathan Warrell, Ali Shahrokni, and Philip H. S. Torr. 2013. Mesh based semantic modelling for indoor and outdoor scenes. In IEEE Conference on Computer Vision and Pattern Recognition. 2067--2074.

Digital Library

[62]

Nguyen Van Sinh, Tran Manh Ha, and Nguyen Tien Thanh. 2016. Filling holes on the surface of 3D point clouds based on tangent plane of hole boundary points. In Proceedings of the Seventh Symposium on Information and Communication Technology. ACM, 331--338.

Digital Library

[63]

Joan Verdera, Vicent Caselles, Marcelo Bertalmio, and Guillermo Sapiro. 2003. Inpainting surface holes. In Proceedings of the International Conference on Image Processing (ICIP), vol. 2. 903--906.

[64]

Paul A. Viola and Michael J. Jones. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 511--518.

[65]

J. Wang and M. M. Oliveira. 2003. A hole-filling strategy for reconstruction of smooth surfaces in range images. In SIBGRAPI. IEEE Computer Society, 11--18.

[66]

Jianning Wang and Manuel M. Oliveira. 2007. Filling holes on locally smooth surfaces reconstructed from point clouds. Image and Vision Computing 25, 1 (2007), 103--113.

[67]

Xiao J. Wu, Michael Y. Wang, and B. Han. 2008. An automatic hole-filling algorithm for polygon meshes. Comput. Aided Des. Appl. 5 (2008), 889--899.

[68]

Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3D shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1912--1920.

[69]

Kai Xu, Vladimir G. Kim, Qixing Huang, Niloy Mitra, and Evangelos Kalogerakis. 2016. Data-driven shape analysis and processing. In SIGGRAPH ASIA 2016 Courses. ACM, Article 4, 38 pages.

Digital Library

[70]

Wei Zhao, Shuming Gao, and Hongwei Lin. 2007. A robust hole-filling algorithm for triangular mesh. Vis. Comput. 23, 12 (2007), 987--997.

Digital Library

Cited By

Tabib RHegde DAnvekar TMudenagudi U(2023)DeFi: Detection and Filling of Holes in Point Clouds Towards Restoration of Digitized Cultural Heritage Models2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)10.1109/ICCVW60793.2023.00175(1595-1604)Online publication date: 2-Oct-2023
https://doi.org/10.1109/ICCVW60793.2023.00175
Basu APaul SGhosh SDas SChanda BBhagvati CSnasel V(2023)Digital Restoration of Cultural Heritage With Data-Driven Computing: A SurveyIEEE Access10.1109/ACCESS.2023.328063911(53939-53977)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3280639
Uddin KJeong TOh B(2022)Incomplete Region Estimation and Restoration of 3D Point Cloud Human Face DatasetsSensors10.3390/s2203072322:3(723)Online publication date: 18-Jan-2022
https://doi.org/10.3390/s22030723
Show More Cited By

Index Terms

Region of Interest-Based 3D Inpainting of Cultural Heritage Artifacts
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
      1. Point-based models
      2. Shape analysis
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Region of interest (ROI) based 3D inpainting
SA '16: SIGGRAPH ASIA 2016 Posters

We address the problem of 3D inpainting using ROI-based method for point cloud data. We focus on inpainting of complex, irregular and large missing regions covering prominent geometric features by considering n self-similar examples. The effectiveness ...
Example-based 3D inpainting of point clouds using metric tensor and Christoffel symbols

In this paper, we address the problem of 3D inpainting using example-based methods for point cloud data. 3D inpainting is a process of filling holes or missing regions in the reconstructed 3D models. Typically inpainting methods addressed in the ...
Discriminative image hashing based on region of interest
MMM'10: Proceedings of the 16th international conference on Advances in Multimedia Modeling

In this paper, we propose a discriminative image hashing scheme based on Region of Interest (ROI) in order to increase the discriminative capability under image content modifications, while the robustness to content preserving operations is also ...

Comments

Information & Contributors

Information

Published In

cover image Journal on Computing and Cultural Heritage

Journal on Computing and Cultural Heritage Volume 11, Issue 2

June 2018

124 pages

ISSN:1556-4673

EISSN:1556-4711

DOI:10.1145/3199679

Editor:
Roberto Scopigno
CNR-ISTI, Italy

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 May 2018

Accepted: 01 July 2017

Revised: 01 June 2017

Received: 01 November 2016

Published in JOCCH Volume 11, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Department of Science and Technology, Govt. of India

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
295
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tabib RHegde DAnvekar TMudenagudi U(2023)DeFi: Detection and Filling of Holes in Point Clouds Towards Restoration of Digitized Cultural Heritage Models2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)10.1109/ICCVW60793.2023.00175(1595-1604)Online publication date: 2-Oct-2023
https://doi.org/10.1109/ICCVW60793.2023.00175
Basu APaul SGhosh SDas SChanda BBhagvati CSnasel V(2023)Digital Restoration of Cultural Heritage With Data-Driven Computing: A SurveyIEEE Access10.1109/ACCESS.2023.328063911(53939-53977)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3280639
Uddin KJeong TOh B(2022)Incomplete Region Estimation and Restoration of 3D Point Cloud Human Face DatasetsSensors10.3390/s2203072322:3(723)Online publication date: 18-Jan-2022
https://doi.org/10.3390/s22030723
Zhou ZLiu XShang JHuang JLi ZJia H(2022)Inpainting Digital Dunhuang Murals with Structure-Guided Deep NetworkJournal on Computing and Cultural Heritage 10.1145/353286715:4(1-25)Online publication date: 25-Jul-2022
https://dl.acm.org/doi/10.1145/3532867
Acke LDe Vis KVerwulgen SVerlinden J(2021)Survey and literature study to provide insights on the application of 3D technologies in objects conservation and restorationJournal of Cultural Heritage10.1016/j.culher.2020.12.003Online publication date: Mar-2021
https://doi.org/10.1016/j.culher.2020.12.003

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents