Monte Carlo Estimators for Differential Light Transport

Source code of the paper "Monte Carlo Estimators for Differential Light Transport" by Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob from SIGGRAPH 2021.

The implementation is based on the Mitsuba 3 renderer, see the lower part of the README.

Compiling

Please see the documentation for details on how to compile this codebase.

Results

The results directory contains a set of Jupyter notebooks that reproduce some of the experiments from the paper.

1. Differential Microfacet Sampling Strategy

   This notebook creates a list of plots to illustrate the differential Microfacet sampling strategies discusses in Section 4.1 of the paper.

2. 1D Reparameterizations

   Simple 1D examples of the reparameterization ideas dicussed in Section 5 of the paper. Illustrates both the cases of ...

   • Parameter-dependent discontinuities from moving geometry.
   • Parameter-dependent discontinuities from static geometry in presence of attached sampling strategies.

3. Estimator comparison (vMF conductor)

   Forward and reverse mode differentiation of a rough conductor roughness parameter in a simple test scene with a vMF emitter. Compares a long list of differential estimators:

   • Detached emitter sampling

   • Detached BSDF sampling

   • Attached BSDF sampling

   • Detached MIS weights, detached BSDF sampling, detached emitter sampling

   • Attached MIS weights, attached BSDF sampling, detached emitter sampling

   • Detached MIS weights, attached BSDF sampling, detached emitter sampling (biased!)

   • Attached MIS weights, detached BSDF sampling, detached emitter sampling

   • Detached BSDF sampling using the differential microfacet sampling strategy

   • Detached MIS weights, detached differential microfacet BSDF sampling, detached emitter sampling

4. Estimator comparison (vMF dielectric)

   Same as (3) but with a rough dielectric material instead.

5. Estimator comparison (envmap conductor)

   Same as (3) but with an environment emitter instead of the vMF.

6. Estimator comparison (envmap dielectric)

   Same as (5) but with a rough dielectric material instead.

7. Estimator comparison (roughness optimization)

   A simple gradient based optimization recovering a roughness texture from a given reference rendering. Compares three different BSDF sampling techniques under two different illumination conditions. (Similar to Figure 12 from the paper):

   • Detached BSDF sampling
   • Attached BSDF sampling
   • Detached BSDF sampling using the differential microfacet sampling strategy

8. Estimator comparison (Veach)

   Reproduces the (forward mode) gradient images of the classical Veach test scene from Figure 11 in the paper. Compared to the simpler test scenes above, this scene contains (static) visibility discontinuities that causes naïve attached estimators to be biased. In addition to the previously listed estimators, two additional ones involving reparameterized attached sampling are used:

   • Reparameterized attached BSDF sampling
   • Attached MIS weights, reparameterized attached BSDF sampling, detached emitter sampling

9. Estimator comparison (attached reparameterized)

   Again illustrates the bias of attached BSDF sampling in presence of static visibility discontinuities, this time visualizing parameter gradient textures produced by reverse mode differentiation. Based on Figure 10 of the paper.

   Additionally includes a second test scene that shows the same issue for discontinous shading normals from a normal map texture.

Mitsuba Renderer 3

Documentation	Tutorial videos	Linux	MacOS	Windows

Introduction

Mitsuba 3 is a research-oriented rendering system for forward and inverse light transport simulation developed at EPFL in Switzerland. It consists of a core library and a set of plugins that implement functionality ranging from materials and light sources to complete rendering algorithms.

Mitsuba 3 is retargetable: this means that the underlying implementations and data structures can transform to accomplish various different tasks. For example, the same code can simulate both scalar (classic one-ray-at-a-time) RGB transport or differential spectral transport on the GPU. This all builds on Dr.Jit, a specialized just-in-time (JIT) compiler developed specifically for this project.

Main Features

• Cross-platform: Mitsuba 3 has been tested on Linux (x86_64), macOS (aarch64, x86_64), and Windows (x86_64).

• High performance: The underlying Dr.Jit compiler fuses rendering code into kernels that achieve state-of-the-art performance using an LLVM backend targeting the CPU and a CUDA/OptiX backend targeting NVIDIA GPUs with ray tracing hardware acceleration.

• Python first: Mitsuba 3 is deeply integrated with Python. Materials, textures, and even full rendering algorithms can be developed in Python, which the system JIT-compiles (and optionally differentiates) on the fly. This enables the experimentation needed for research in computer graphics and other disciplines.

• Differentiation: Mitsuba 3 is a differentiable renderer, meaning that it can compute derivatives of the entire simulation with respect to input parameters such as camera pose, geometry, BSDFs, textures, and volumes. It implements recent differentiable rendering algorithms developed at EPFL.

• Spectral & Polarization: Mitsuba 3 can be used as a monochromatic renderer, RGB-based renderer, or spectral renderer. Each variant can optionally account for the effects of polarization if desired.

Tutorial videos, documentation

We've recorded several YouTube videos that provide a gentle introduction Mitsuba 3 and Dr.Jit. Beyond this you can find complete Juypter notebooks covering a variety of applications, how-to guides, and reference documentation on readthedocs.

About

This project was created by Wenzel Jakob. Significant features and/or improvements to the code were contributed by Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang.