skip to main content
research-article

Fast volume rendering with spatiotemporal reservoir resampling

Published: 10 December 2021 Publication History

Abstract

Volume rendering under complex, dynamic lighting is challenging, especially if targeting real-time. To address this challenge, we extend a recent direct illumination sampling technique, spatiotemporal reservoir resampling, to multi-dimensional path space for volumetric media.
By fully evaluating just a single path sample per pixel, our volumetric path tracer shows unprecedented convergence. To achieve this, we properly estimate the chosen sample's probability via approximate perfect importance sampling with spatiotemporal resampling. A key observation is recognizing that applying cheaper, biased techniques to approximate scattering along candidate paths (during resampling) does not add bias when shading. This allows us to combine transmittance evaluation techniques: cheap approximations where evaluations must occur many times for reuse, and unbiased methods for final, per-pixel evaluation.
With this reformulation, we achieve low-noise, interactive volumetric path tracing with arbitrary dynamic lighting, including volumetric emission, and maintain interactive performance even on high-resolution volumes. When paired with denoising, our low-noise sampling helps preserve smaller-scale volumetric details.

Supplementary Material

ZIP File (a279-lin.zip)
Supplemental files.
MP4 File (a279-lin.mp4)

References

[1]
Philippe Bekaert, Mateu Sbert, and John H Halton. 2002. Accelerating Path Tracing by Re-Using Paths. In Rendering Techniques. 125--134.
[2]
Nir Benty, Kai-Hwa Yao, Petrik Clarberg, Lucy Chen, Simon Kallweit, Tim Foley, Matthew Oakes, Conor Lavelle, and Chris Wyman. 2020. The Falcor Rendering Framework. https://github.com/NVIDIAGameWorks/Falcor
[3]
Benedikt Bitterli. 2021. Correlations and reuse for fast and accurate physically based light transport. Ph.D. Dissertation. Dartmouth College.
[4]
Benedikt Bitterli and Wojciech Jarosz. 2017. Beyond points and beams: Higher-dimensional photon samples for volumetric light transport. ACM Trans. Graph. 36, 4 (2017), 1--12.
[5]
Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting. ACM Trans. Graph. 39, 4 (2020), 148:1--148:17.
[6]
Jakub Boksansky, Paula Jukarainen, and Chris Wyman. 2021. Rendering Many Lights with Grid-Based Reservoirs. In Ray Tracing Gems II. Springer, 351--365.
[7]
Min-Te Chao. 1982. A general purpose unequal probability sampling plan. Biometrika 69, 3 (1982), 653--656.
[8]
Cyril Delalandre, Pascal Gautron, Jean-Eudes Marvie, and Guillaume François. 2011. Transmittance function mapping. In Symposium on Interactive 3D Graphics and Games. 31--38.
[9]
Xi Deng, Shaojie Jiao, Benedikt Bitterli, and Wojciech Jarosz. 2019. Photon surfaces for robust, unbiased volumetric density estimation. ACM Trans. Graph. 38, 4 (2019), 46.
[10]
Eugene d'Eon and Jan Novák. 2021. Zero-variance Transmittance Estimation. In Eurographics Symposium on Rendering - DL-only Track.
[11]
Mathieu Galtier, Stephane Blanco, Cyril Caliot, Christophe Coustet, Jérémi Dauchet, Mouna El Hafi, Vincent Eymet, Richard Fournier, Jacques Gautrais, Anais Khuong, et al. 2013. Integral formulation of null-collision Monte Carlo algorithms. Journal of Quantitative Spectroscopy and Radiative Transfer 125 (2013), 57--68.
[12]
Pascal Gautron, Cyril Delalandre, Jean-Eudes Marvie, and Pascal Lecocq. 2013. Boundary-aware extinction mapping. Computer Graphics Forum 32, 7 (2013), 305--314.
[13]
Iliyan Georgiev, Jaroslav Krivanek, Toshiya Hachisuka, Derek Nowrouzezahrai, and Wojciech Jarosz. 2013. Joint importance sampling of low-order volumetric scattering. ACM Trans. Graph. 32, 6 (2013), 164--1.
[14]
Iliyan Georgiev, Zackary Misso, Toshiya Hachisuka, Derek Nowrouzezahrai, Jaroslav Křivánek, and Wojciech Jarosz. 2019. Integral formulations of volumetric transmittance. ACM Trans. Graph. 38, 6 (2019), 1--17.
[15]
Adrien Gruson, Binh-Son Hua, Nicolas Vibert, Derek Nowrouzezahrai, and Toshiya Hachisuka. 2018. Gradient-domain volumetric photon density estimation. ACM Trans. Graph. 37, 4 (2018), 1--13.
[16]
Louis G Henyey and Jesse L Greenstein. 1941. Diffuse radiation in the galaxy. Astro-physical Journal 93 (1941), 70--83.
[17]
Sebastian Herholz, Yangyang Zhao, Oskar Elek, Derek Nowrouzezahrai, Hendrik PA Lensch, and Jaroslav Křivánek. 2019. Volume path guiding based on zero-variance random walk theory. ACM Trans. Graph. 38, 3 (2019), 1--19.
[18]
Sébastien Hillaire. 2015. Physically-based and Unified Volumetric Rendering in Frostbite. In SIGGRAPH Courses: Advances in Real-Time Rendering in Games.
[19]
Rama Karl Hoetzlein. 2016. GVDB: Raytracing Sparse Voxel Database Structures on the GPU. In High Performance Graphics. 109--117.
[20]
Nikolai Hofmann, Jon Hasselgren, Petrik Clarberg, and Jacob Munkberg. 2021. Interactive Path Tracing and Reconstruction of Sparse Volumes. Proc. ACM Comput. Graph. Interact. Tech. 4, 1, Article 5 (April 2021), 19 pages.
[21]
Jon Jansen and Louis Bavoil. 2010. Fourier opacity mapping. In Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games. 165--172.
[22]
Wojciech Jarosz, Derek Nowrouzezahrai, Iman Sadeghi, and Henrik Wann Jensen. 2011. A comprehensive theory of volumetric radiance estimation using photon points and beams. ACM Trans. Graph. 30, 1 (2011), 1--19.
[23]
Wojciech Jarosz, Matthias Zwicker, and Henrik Wann Jensen. 2008. The beam radiance estimate for volumetric photon mapping. In ACM SIGGRAPH 2008 classes. 1--112.
[24]
Henrik Wann Jensen and Per H Christensen. 1998. Efficient simulation of light transport in scenes with participating media using photon maps. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 311--320.
[25]
Anton Kaplanyan and Carsten Dachsbacher. 2010. Cascaded Light Propagation Volumes for Real-Time Indirect Illumination. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 99--107.
[26]
Markus Kettunen, Eugene D'Eon, Jacopo Pantaleoni, and Jan Novák. 2021. An Unbiased Ray-Marching Transmittance Estimator. ACM Trans. Graph. 40, 4, Article 137 (July 2021), 20 pages.
[27]
Markus Kettunen, Marco Manzi, Miika Aittala, Jaakko Lehtinen, Frédo Durand, and Matthias Zwicker. 2015. Gradient-Domain Path Tracing. ACM Trans. Graph. 34, 4, Article 123 (July 2015), 13 pages.
[28]
Emmett Kilgariff, Henry Moreton, Nick Stam, and Brandon Bell. 2018. NVIDIA Turing Architecture In-Depth. https://developer.nvidia.com/blog/nvidia-turing-architecture-in-depth/. [Online; accessed 19-January-2021].
[29]
Tae-Yong Kim and Ulrich Neumann. 2001. Opacity shadow maps. In Rendering Techniques 2001. Springer, 177--182.
[30]
Jaroslav Křivánek and Eugene d'Eon. 2014. A zero-variance-based sampling scheme for Monte Carlo subsurface scattering. In ACM SIGGRAPH 2014 Talks. 1--1.
[31]
Jaroslav Křivánek, Iliyan Georgiev, Toshiya Hachisuka, Petr Vévoda, Martin Šik, Derek Nowrouzezahrai, and Wojciech Jarosz. 2014. Unifying points, beams, and paths in volumetric light transport simulation. ACM Trans. Graph. 33, 4 (2014), 1--13.
[32]
Christopher Kulla and Marcos Fajardo. 2012. Importance sampling techniques for path tracing in participating media. Computer graphics forum 31, 4 (2012), 1519--1528.
[33]
Peter Kutz, Ralf Habel, Yining Karl Li, and Jan Novák. 2017. Spectral and decomposition tracking for rendering heterogeneous volumes. ACM Trans. Graph. 36, 4 (2017), 1--16.
[34]
Jaakko Lehtinen, Tero Karras, Samuli Laine, Miika Aittala, Frédo Durand, and Timo Aila. 2013. Gradient-Domain Metropolis Light Transport. 32, 4, Article 95 (July 2013), 12 pages.
[35]
Zander Majercik, Jean-Philippe Guertin, Derek Nowrouzezahrai, and Morgan McGuire. 2019. Dynamic Diffuse Global Illumination with Ray-Traced Irradiance Fields. Journal of Computer Graphics Techniques (JCGT) 8, 2 (5 June 2019), 1--30.
[36]
Johannes Meng, Johannes Hanika, and Carsten Dachsbacher. 2016. Improving the Dwivedi sampling scheme. Computer Graphics Forum 35, 4 (2016), 37--44.
[37]
Bailey Miller, Iliyan Georgiev, and Wojciech Jarosz. 2019. A null-scattering path integral formulation of light transport. ACM Trans. Graph. 38, 4 (2019), 1--13.
[38]
Pierre Moreau, Matt Pharr, Petrik Clarberg, M Steinberger, and T Foley. 2019. Dynamic Many-Light Sampling for Real-Time Ray Tracing. In High Performance Graphics (Short Papers). 21--26.
[39]
Ken Museth. 2013. VDB: High-Resolution Sparse Volumes with Dynamic Topology. ACM Trans. Graph. 32, 3, Article 27 (2013), 22 pages.
[40]
Jan Novák, Derek Nowrouzezahrai, Carsten Dachsbacher, and Wojciech Jarosz. 2012a. Progressive virtual beam lights. Computer Graphics Forum 31, 4 (2012), 1407--1413.
[41]
Jan Novák, Derek Nowrouzezahrai, Carsten Dachsbacher, and Wojciech Jarosz. 2012b. Virtual ray lights for rendering scenes with participating media. ACM Trans. Graph. 31, 4 (2012), 1--11.
[42]
Jan Novák, Andrew Selle, and Wojciech Jarosz. 2014. Residual ratio tracking for estimating attenuation in participating media. ACM Trans. Graph. 33, 6 (2014), 179--1.
[43]
Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo methods for volumetric light transport simulation. Computer Graphics Forum (Proceedings of Eurographics - State of the Art Reports) 37, 2 (May 2018).
[44]
NVIDIA. 2017. NVIDIA® OptiX™ AI-Accelerated Denoiser. https://developer.nvidia.com/optix-denoiser
[45]
Yaobin Ouyang, Shiqiu Liu, Markus Kettunen, Matt Pharr, and Jacopo Pantaleoni. 2021. ReSTIR GI: Path Resampling for Real-Time Path Tracing. Computer Graphics Forum (2021).
[46]
Matthias Raab, Daniel Seibert, and Alexander Keller. 2008. Unbiased global illumination with participating media. In Monte Carlo and Quasi-Monte Carlo Methods 2006. Springer, 591--605.
[47]
Marco Salvi, Kiril Vidimče, Andrew Lauritzen, and Aaron Lefohn. 2010. Adaptive volumetric shadow maps. Computer Graphics Forum 29, 4 (2010), 1289--1296. Florian Simon, Johannes Hanika, Tobias Zirr, and Carsten Dachsbacher. 2017. Line Integration for Rendering Heterogeneous Emissive Volumes. Computer Graphics Forum 36 (July 2017), 101--110.
[48]
TM Sutton, FB Brown, FG Bischoff, DB MacMillan, CL Ellis, JT Ward, CT Ballinger, DJ Kelly, and L Schindler. 1999. The physical models and statistical procedures used in the RACER Monte Carlo code. Technical Report. Knolls Atomic Power Lab.
[49]
László Szirmay-Kalos, Iliyan Georgiev, Milán Magdics, Balazs Molnar, and David Legrady. 2017. Unbiased Light Transport Estimators for Inhomogeneous Participating Media. Computer Graphics Forum 36 (May 2017), 9--19.
[50]
László Szirmay-Kalos, Balázs Tóth, and Milán Magdics. 2011. Free Path Sampling in High Resolution Inhomogeneous Participating Media. Computer Graphics Forum 30 (March 2011), 85--97.
[51]
Justin Talbot, David Cline, and Parris Egbert. 2005. Importance Resampling for Global Illumination. In Eurographics Symposium on Rendering. 139--146.
[52]
Justin F Talbot. 2005. Importance resampling for global illumination. Master's thesis. Brigham Young University.
[53]
Jiří Vorba, Johannes Hanika, Sebastian Herholz, Thomas Müller, Jaroslav Křivánek, and Alexander Keller. 2019. Path Guiding in Production. In ACM SIGGRAPH 2019 Courses (Los Angeles, California). ACM, New York, NY, USA, Article 18, 77 pages.
[54]
E Woodcock, T Murphy, P Hemmings, and S Longworth. 1965. Techniques used in the GEM code for Monte Carlo neutronics calculations in reactors and other systems of complex geometry. In Proc. Conf. Applications of Computing Methods to Reactor Problems, Vol. 557.
[55]
Chris Wyman. 2016. Exploring and Expanding the Continuum of OIT Algorithms. In High Performance Graphics. 1--11. Chris Wyman and Alexey Panteleev. 2021. Rearchitecting Spatiotemporal Resampling for Production. In High-Performance Graphics - Symposium Papers.
[56]
Yonghao Yue, Kei Iwasaki, B. Chen, Y. Dobashi, and T. Nishita. 2011. Toward Optimal Space Partitioning for Unbiased, Adaptive Free Path Sampling of Inhomogeneous Participating Media. Computer Graphics Forum 30 (2011).
[57]
Cem Yuksel and John Keyser. 2008. Deep Opacity Maps. Computer Graphics Forum (Proceedings of EUROGRAPHICS 2008) 27, 2 (2008), 675--680.

Cited By

View all

Index Terms

  1. Fast volume rendering with spatiotemporal reservoir resampling

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 40, Issue 6
    December 2021
    1351 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/3478513
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 10 December 2021
    Published in TOG Volume 40, Issue 6

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. ReSTIR
    2. ray tracing
    3. resampled importance sampling
    4. reservoir sampling
    5. volume rendering

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)110
    • Downloads (Last 6 weeks)8
    Reflects downloads up to 09 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media