Skip to content

Latest commit

 

History

History
61 lines (52 loc) · 3.9 KB

README.md

File metadata and controls

61 lines (52 loc) · 3.9 KB
Raw

ꟻLIP: A Tool for Visualizing and Communicating Errors in Rendered Images (v1.3)

By Pontus Andersson, Jim Nilsson, and Tomas Akenine-Möller, with Magnus Oskarsson, Kalle Åström, Mark D. Fairchild, and Peter Shirley.

This repository holds implementations of the LDR-ꟻLIP and HDR-ꟻLIP image error metrics as loss modules in PyTorch.

License

Copyright © 2020-2024, NVIDIA Corporation & Affiliates. All rights reserved.

This work is made available under a BSD 3-Clause License.

The repository distributes code for tinyexr, which is subject to a BSD 3-Clause License,
and stb_image, which is subject to an MIT License.

For individual contributions to the project, please confer the Individual Contributor License Agreement.

For business inquiries, please visit our website and submit the form: NVIDIA Research Licensing.

PyTorch (Loss Function)

  • Setup (with Anaconda3): 
    conda create -n flip_dl python numpy matplotlib
conda activate flip_dl
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge
conda install -c conda-forge openexr-python
  • Remember to activate the flip_dl environment through conda activate flip_dl before using the loss function.
  • LDR- and HDR-ꟻLIP are implemented as loss modules in flip_loss.py. An example where the loss function is used to train a simple autoencoder is provided in train.py.
  • Tested on Windows with Conda 4.10.0, CUDA 11.2, Python 3.9.4, PyTorch 1.8.1, NumPy 1.20.1, and OpenEXR b1.3.2.
  • Per default, the loss function returns the mean of the error maps. To return the full error maps, remove torch.mean() from the forward() function.
  • For LDR-ꟻLIP, the images are assumed to be in sRGB space (change the color space transform in LDRFLIPLoss's forward() function to linrgb2ycxcz if your network's output is in linear RGB), in the [0,1] range.
  • Both LDR- and HDR-ꟻLIP takes an optional argument describing the assumed number of pixels per degree of the observer. Per default, it is assume that the images are viewed at a distance 0.7 m from a 0.7 m wide 4K monitor.
  • The HDRFLIPLoss can take three additional, optional arguments: tone_mapper, start_exposure, and stop_exposure. tone_mapper is a string describing the tone mapper that HDR-ꟻLIP should assume, for which the choices are aces (default), hable, and reinhard. The default assumption is the ACES tone mapper. start_exposure, and stop_exposure should have Nx1x1x1 layout and hold the start and stop exposures, respectively, used for each of the N reference/test pairs in the batch. Per default, HDRFLIPLoss computes start and stop exposures as described in the paper. NOTE: When start and/or stop exposures are not provided, HDR-ꟻLIP is not symmetric. The user should therefore make sure to input the test images as the first argument and the reference image as the second argument to the HDRFLIPLoss's forward() function.
  • ../tests/test_pytorch.py contains simple tests used to test whether code updates alter results and data.py contains image loading/saving functions.