Sea of images

A long-standing research problem in computer graphics is to reproduce the visual experience of walking through a large photorealistic environment interactively. On one hand, traditional geometry-based rendering systems fall short of simulating the visual realism of a complex environment. On the other hand, image-based rendering systems have to date been unable to capture and store a sampled representation of a large environment with complex lighting and visibility effects.In this paper, we present a "Sea of Images," a practical approach to dense sampling, storage, and reconstruction of the plenoptic function in large, complex indoor environments. We use a motorized cart to capture omnidirectional images every few inches on a eye-height plane throughout an environment. The captured images are compressed and stored in a multiresolution hierarchy suitable for real-time prefetching during an interactive walkthrough. Later, novel images are reconstructed for a simulated observer by resampling nearby captured images.Our system acquires 15,254 images over 1,050 square feet at an average image spacing of 1.5 inches. The average capture and processing time is 7 hours. We demonstrate realistic walkthroughs of real-world environments reproducing specular reflections and occlusion effects while rendering 15-25 frames per second.