research-article

OmniKinect: real-time dense volumetric data acquisition and applications

Authors:

Bernhard Kainz,

Stefan Hauswiesner,

Gerhard Reitmayr,

Markus Steinberger,

Raphael Grasset,

Denis Kalkofen,

Hartmut Seichter,

Dieter SchmalstiegAuthors Info & Claims

VRST '12: Proceedings of the 18th ACM symposium on Virtual reality software and technology

Pages 25 - 32

https://doi.org/10.1145/2407336.2407342

Published: 10 December 2012 Publication History

Abstract

Real-time three-dimensional acquisition of real-world scenes has many important applications in computer graphics, computer vision and human-computer interaction. Inexpensive depth sensors such as the Microsoft Kinect allow to leverage the development of such applications. However, this technology is still relatively recent, and no detailed studies on its scalability to dense and view-independent acquisition have been reported. This paper addresses the question of what can be done with a larger number of Kinects used simultaneously. We describe an interference-reducing physical setup, a calibration procedure and an extension to the KinectFusion algorithm, which allows to produce high quality volumetric reconstructions from multiple Kinects whilst overcoming systematic errors in the depth measurements. We also report on enhancing image based visual hull rendering by depth measurements, and compare the results to KinectFusion. Our system provides practical insight into achievable spatial and radial range and into bandwidth requirements for depth data acquisition. Finally, we present a number of practical applications of our system.

Supplementary Material

JPG File (vrst0105-file3.jpg)

Download
10.79 KB

suppl.mov (vrst0105-file3.mp4)

Supplemental video

Download
42.28 MB

References

[1]

K. Berger, K. Ruhl, C. Brümmer, Y. Schröder, A. Scholz, and M. Magnor. Markerless motion capture using multiple color-depth sensors. In Proc. VMV 2011, pages 317--324, Oct. 2011.

[2]

E. Boyer. A hybrid approach for computing visual hulls of complex objects. In Proc. IEEE CVPR, pages 695--701, 2003.

Digital Library

[3]

A. Butler, S. Izadi, O. Hilliges, D. Molyneaux, S. Hodges, and D. Kim. Shake'n'sense: reducing interference for overlapping structured light depth cameras. In Proc. CHI '12, pages 1933--1936, 2012.

Digital Library

[4]

B. Curless and M. Levoy. A volumetric method for building complex models from range images. In Proc. SIGGRAPH '96, pages 303--312, 1996.

Digital Library

[5]

U. Dhond and J. Aggarwal. Structure from stereo - a review. IEEE Trans. on Systems, Man and Cybernetics, 19(6):1489 --1510, nov/dec 1989.

[6]

L. Guan, J.-S. Franco, and M. Pollefeys. 3D Object Reconstruction with Heterogeneous Sensor Data. In Proc. 3DPVT, 2008.

[7]

D. Kalkofen, E. Mendez, and D. Schmalstieg. Comprehensible visualization for augmented reality. IEEE TVCG, 15(2):193--204, 2009.

Digital Library

[8]

Y. M. Kim, D. Chan, C. Theobalt, and S. Thrun. Design and calibration of a multi-view TOF sensor fusion system. In Proc. IEEE CVPR Workshops, pages 1--7, June 2008.

[9]

C. Kuster, T. Popa, C. Zach, C. Gotsman, and M. Gross. Freecam: A hybrid camera system for interactive free-viewpoint video. In Proc. VMV, 2011.

[10]

M. Li. Towards Real-Time Novel View Synthesis Using Visual Hulls. PhD thesis, Universite at des Saarlandes, 2004.

[11]

A. Maimone and H. Fuchs. Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras. In Proc. ISMAR '11, pages 137--146, 2011.

Digital Library

[12]

A. Maimone and H. Fuchs. Reducing interference between multiple structured light depth sensors using motion. In Proc. IEEE VR, pages 51--54, 2012.

Digital Library

[13]

W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. McMillan. Image-based visual hulls. In Proc. ACM SIGGRAPH '00, pages 369--374, 2000.

Digital Library

[14]

W. Matusik and H. Pfister. 3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. In Proc. SIGGRAPH '04, pages 814--824, 2004.

Digital Library

[15]

R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In Proc IEEE ISMAR '11, pages 127--136, 2011.

Digital Library

[16]

S. Prince, A. D. Cheok, F. Farbiz, T. Williamson, N. Johnson, M. Billinghurst, and H. Kato. 3D Live: Real Time Captured Content for Mixed Reality. In Proc. IEEE ISMAR '02, page 7, 2002.

Digital Library

[17]

C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz. Fast cost-volume filtering for visual correspondence and beyond. In Proc. IEEE CVPR '11, pages 3017--3024, 2011.

Digital Library

[18]

J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake. Real-time human pose recognition in parts from single depth images. In Proc. IEEE CVPR'11, 2011.

Digital Library

[19]

J. Tong, J. Zhou, L. Liu, Z. Pan, and H. Yan. Scanning 3d full human bodies using kinects. IEEE TVCG, pages 643--650, 2012.

Digital Library

[20]

D. Wagner, T. Langlotz, and D. Schmalstieg. Robust und unobtrusive marker tracking on mobile phones. In Proc. IEEE ISMAR'08, pages 121--124, 2008.

Digital Library

[21]

A. D. Wilson and H. Benko. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In Proc. ACM UIST '10, pages 273--282, 2010.

Digital Library

[22]

R. Yang, G. Welch, and G. Bishop. Real-time consensus-based scene reconstruction using commodity graphics hardware. In Proc. Computer Graphics and Applications 2002, pages 225--234, 2002.

Digital Library

Cited By

Wu GZhang HYan ZWang HZhong ZYin Z(2024)3D pipeline reconstruction and diameter measurement method based on target segmentationInternational Conference on Algorithm, Imaging Processing, and Machine Vision (AIPMV 2023)10.1117/12.3014403(40)Online publication date: 9-Jan-2024
https://doi.org/10.1117/12.3014403
Kamble TMahajan S(2022)3D Vision Using Multiple Structured Light-Based Kinect Depth CamerasInternational Journal of Image and Graphics10.1142/S021946782450001324:01Online publication date: 21-Sep-2022
https://doi.org/10.1142/S0219467824500013
Druta RDruta CNegirla PSilea I(2021)A Review on Methods and Systems for Remote CollaborationApplied Sciences10.3390/app11211003511:21(10035)Online publication date: 26-Oct-2021
https://doi.org/10.3390/app112110035
Show More Cited By

Index Terms

OmniKinect: real-time dense volumetric data acquisition and applications

Recommendations

Pose Depth Volume extraction from RGB-D streams for frontal gait recognition

Highlights We combine depth and RGB information from Kinect for frontal gait recognition. Key poses are extracted using depth frames registered in RGB frame coordinate system. A new feature named Pose Depth Volume is proposed. Comparative study with ...
Compression of Noisy Depth Image using Planes
ICVGIP '14: Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image Processing

This paper investigates compression of depth images – particularly, noisy depth images – captured by depth sensors like Kinect. Our scheme is based on incrementally detecting planes in depth images and then storing the plane parameters instead of the ...
Fusion4D: real-time performance capture of challenging scenes

We contribute a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time. Our algorithm supports both incremental reconstruction, improving the surface estimation over time, as well ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '12: Proceedings of the 18th ACM symposium on Virtual reality software and technology

December 2012

226 pages

ISBN:9781450314695

DOI:10.1145/2407336

General Chair:
Mark Green
University of Ontario Institute of Technology, Canada
,
Program Chairs:
Wolfgang Stuerzlinger
York University, Toronto, Canada
,
Marc Erich Latoschik
University of Würzburg, Germany
,
Bill Kapralos
University of Ontario Institute of Technology, Canada

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 December 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VRST'12

Sponsor:

VRST'12: The 18th ACM Symposium on Virtual Reality Software and Technology 2012

December 10 - 12, 2012

Ontario, Toronto, Canada

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
915
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu GZhang HYan ZWang HZhong ZYin Z(2024)3D pipeline reconstruction and diameter measurement method based on target segmentationInternational Conference on Algorithm, Imaging Processing, and Machine Vision (AIPMV 2023)10.1117/12.3014403(40)Online publication date: 9-Jan-2024
https://doi.org/10.1117/12.3014403
Kamble TMahajan S(2022)3D Vision Using Multiple Structured Light-Based Kinect Depth CamerasInternational Journal of Image and Graphics10.1142/S021946782450001324:01Online publication date: 21-Sep-2022
https://doi.org/10.1142/S0219467824500013
Druta RDruta CNegirla PSilea I(2021)A Review on Methods and Systems for Remote CollaborationApplied Sciences10.3390/app11211003511:21(10035)Online publication date: 26-Oct-2021
https://doi.org/10.3390/app112110035
Regenbrecht HOtt CPark NDuncan SCollins J(2021)Voxelvideos for Entertainment, Education, and TrainingIEEE Access10.1109/ACCESS.2021.30764889(68185-68196)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3076488
Lilija KPohl HHornbæk KBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Who Put That There? Temporal Navigation of Spatial Recordings by Direct ManipulationProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376604(1-11)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376604
Liu ZGong GLi NYu Z(2020)3D reconstruction of human head based on consumer RGB-D sensorsInternational Journal of Modeling, Simulation, and Scientific Computing10.1142/S179396232050061011:06(2050061)Online publication date: 12-Dec-2020
https://doi.org/10.1142/S1793962320500610
Lilija KPohl HBoring SHornbæk KBrewster SFitzpatrick GCox AKostakos V(2019)Augmented Reality Views for Occluded InteractionProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300676(1-12)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300676
Duncan SRegenbrecht HLanglotz T(2019)A Fast Multi-RGBD-Camera Calibration2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR.2019.8798073(906-907)Online publication date: Mar-2019
https://doi.org/10.1109/VR.2019.8798073
Alhwarin FSchiffer SFerrein AScholl I(2019)An Optimized Method for 3D Body Scanning Applications Based on KinectFusionBiomedical Engineering Systems and Technologies10.1007/978-3-030-29196-9_6(100-113)Online publication date: 13-Aug-2019
https://doi.org/10.1007/978-3-030-29196-9_6
Meruvia-Pastor O(2019)Enhancing 3D Capture with Multiple Depth Camera Systems: A State-of-the-Art ReportRGB-D Image Analysis and Processing10.1007/978-3-030-28603-3_7(145-166)Online publication date: 27-Oct-2019
https://doi.org/10.1007/978-3-030-28603-3_7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents