A screen space method of achieving global illumination using convolutional neural networks.
Preprint
Code (Jupyter Notebook)
The Jupyter Notebook should be used in Google Colab.
The dataset used in this work is located in the Blender_Datasets folder.
A pre-trained model for the dataset is located in the Models folder.
After some small changes, the new PSNR of the provided model on unseen images (validation set) is 34.3 dB, compared to 33.8 dB in the preprint.
- Path Tracer: 835ms
- FSSGI: 9ms (92x Speedup)
From Left to Right: Local Illumination (Input) / Predicted GI (Output) / Ground Truth (Path Tracer @1000spp)
Rendering algorithms such as path tracing[1] are commonly used in practice to produce photorealistic images from a 3D scene. However, for complex scenes involving a lot of indirect lighting, path tracing is very computationally expensive and often produces a noisy image when the time budget is limited[2]. The current state of the art method for accelerating rendering time uses a deep convolutional neural network to denoise an image which has been sampled at fewer than 10 samples per pixel[3].
We present a image-translation model that removes the path tracing step entirely and only use a convolutional neural network to predict, in screen space, a visually pleasing photorealistic image using information obtained from the g-buffer. This method is similar to deferred shading, but instead of a human built shader, the 'shader' is created using a machine learning algorithm that learns to reproduce global illumination effects seen in path tracing using the g-buffer data. This method aims to reproduce the quality of path traced images in real-time for conventional rendering pipelines, and can also be used in conjunction with path tracing and denoising techniques as the image generated by the network can serve as an initial estimation which can be refined by more accurate methods.
Deep Shading: Convolutional Neural Networks for Screen-Space Shading
Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network
Photographic Image Synthesis with Cascaded Refinement Networks
[1]KAJIYA J.: The Rendering Equation, 1986
[2]KULLA C., FAJARDO M.: Importance Sampling Techniques for Path Tracing in Participating Media, 2012
[3]CHAITANYA C., KAPLANYAN A., ET AL.: Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder, 2017