How to Set Up a 32-Channel BRDF in Spectral Rendering

[free7187]

Thanks to the discussion#1309 , I can now attempt to edit spectral multi-channel rendering in Python.

However, the working mode for spectral channels is not quite what I expected. To test whether I could edit individual channels of the spectral BRDF, I ran a simple test with the following code:

import sys
sys.path.insert(0, "./mitsuba3/build/Release/python")
import mitsuba as mi
import drjit as dr 
import torch
import torch.nn.functional as F
import torch.nn as nn
import torch.optim as optim
import numpy as np
import os
os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE"
mi.set_variant('cuda_ad_spectral')
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
class real_data_integrator(mi.SamplingIntegrator):
    def __init__(self, props):
        super().__init__(props)
    def sample(self, 
               scene: mi.Scene, 
               sampler:mi.Sampler, 
               ray: mi.RayDifferential3f, 
               medium: mi.Medium = None,
               active: bool = True) -> tuple[mi.Color3f, bool, list[float]]:  
        L = mi.Spectrum(0.0)
        si = scene.ray_intersect(ray, active)        
        valid_ray = active & si.is_valid()
        active &= si.is_valid()
        ds, em_weight = scene.sample_emitter_direction(
            si, sampler.next_2d(), True, active
        )
        active &= ds.pdf != 0.0
        brdf = mi.Spectrum(0.0)
        new_brdf= mi.Spectrum(self.set_value(brdf))
        L += new_brdf *em_weight
        return dr.select(True, L, 0.0), valid_ray, []
    
    @dr.wrap(source='drjit', target='torch')
    def set_value(self,brdf):
        new_brdf=brdf
        new_brdf[0:10,:]=1
        # new_brdf[10:20,:]=1
        # new_brdf[20:30,:]=1
        # new_brdf[:,:]=1
        # new_brdf[20:21,:]=1
        return new_brdf

mi.register_integrator('My_integrator', lambda props: real_data_integrator(props))

xml_path= "spectrum_test.xml"
scene = mi.load_file(xml_path)
with dr.suspend_grad():
    with torch.no_grad():
        image=mi.render(scene)
print("rendering done")
mi.util.write_bitmap("test_spectrum0.png", image)
mi.util.write_bitmap("test_spectrum0.exr", image)

And here is the XML scene setup:

<scene version="3.0.0">
    <default name="spp" value="512"/>
    <default name="res" value="512"/>
    <integrator type="My_integrator"/>

    <sensor type="perspective" id="sensor">
        <string name="fov_axis" value="smaller"/>
        <float name="near_clip" value="0.001"/>
        <float name="far_clip" value="100.0"/>
        <float name="focus_distance" value="1000"/>
        <float name="fov" value="40"/>
        <transform name="to_world">
            <lookat origin="0,  0,  3.6"
                    target="0,  0,  0"
                    up    ="0,  1,  0"/>
        </transform>
        <sampler type="independent">
            <integer name="sample_count" value="$spp"/>
        </sampler>
        <film type="hdrfilm">
            <integer name="width"  value="$res"/>
            <integer name="height" value="$res"/>
            <rfilter type="tent"/>
            <string name="pixel_format" value="rgb"/>
            <string name="component_format" value="float32"/>
        </film>
    </sensor>
<emitter type="directional">
    <vector name="direction" value="0.0, 0.0, -1.0"/>
    <rgb name="irradiance" value="1.0"/>
</emitter>
    <shape type="obj">
		<string name="filename" value="cube.obj" />
		<transform name="to_world">
            <scale value="1"/>
            <translate x="0" y="0" z="0"/>
        </transform>
        <bsdf type="diffuse">
            <rgb name="reflectance" value="1.0"/>
        </bsdf>
	</shape>
</scene>

I initially assumed that with the 32-channel setup, setting channels [0:10], [10:20], and [20:30] of new_brdf to 1 (with other channels set to 0) would result in different colors for the render output. However, the final render always appears gray:

I also tried setting all BRDF channels to 1, as well as setting only one specific channel to 1. The render results are as follows:


After analyzing the RGB ratios for all pixels in these three images, I noticed that they all tend towards (1:1:1). It appears that changing channel values only affects the final output’s brightness, which is quite different from what I had anticipated. I originally expected, for example, that setting channels [10:20] to 1 and others to 0 would result in a green color.

Am I missing something? How should I properly edit spectral BRDF channels to achieve the intended effect?

[merlinND]

Hello @free7187,

Did you check the values of ray.wavelengths? I don't remember the details, but wavelengths may not be sampled in increasing order, and so setting some channels to a certain values could affect wavelengths that you don't expect.
Note also that by default, each ray samples different wavelengths.

PS: when debugging things like this, I would recommend removing torch usage and finding the simplest possible test case, because the more components you add, the more confusing it can be to interpret the results.

[merlinND]

This is the method of the Sensor class that's used to sample wavelengths when creating rays:

mitsuba3/src/render/sensor.cpp

Lines 112 to 127 in af8f6fd

	MI_VARIANT std::pair<typename Sensor<Float, Spectrum>::Wavelength, Spectrum>
	Sensor<Float, Spectrum>::sample_wavelengths(const SurfaceInteraction3f& /*si*/, Float sample,
	Mask active) const {
	if constexpr (is_spectral_v<Spectrum>) {
	if (m_srf != nullptr) {
	return m_srf->sample_spectrum(
	dr::zeros<SurfaceInteraction3f>(),
	math::sample_shifted<Wavelength>(sample),
	active);
	}
	} else {
	DRJIT_MARK_USED(active);
	}
	return sample_wavelength<Float, Spectrum>(sample);
	}

There's a section about SRFs in the docs: https://mitsuba.readthedocs.io/en/latest/src/generated/plugins_sensors.html ("Spectral sensitivities").