the number of channels fixed in the sample function in spectral mode?

[free7187]

Summary

I'm debugging rendering in spectral mode in Python and have set

mi.set_variant('cuda_ad_rgb')

Initially, I configured a BSDF with 75-channel reflectance. However, during the debugging process, I noticed that the return value of bsdf_sample() only has 4 channels. Based on this, I also tried a multi-channel emitter, but the em_weight returned by sample_emitter() also has 4 channels.

This behavior differs from my original expectation. So, during rendering in spectral mode, is the number of channels in the sample function fixed at 4 channels, even if the material parameters may be set to have more channels?

System configuration

System information:

OS: Windows-10
CPU: Intel64 Family 6 Model 183 Stepping 1, GenuineIntel
GPU: NVIDIA GeForce RTX 4090
Python: 3.10.14 | packaged by Anaconda, Inc. | (main, Mar 21 2024, 16:20:14) [MSC v.1916 64 bit (AMD64)]
NVidia driver: 536.25
CUDA: 11.8.89
LLVM: 15.-1.-1

Dr.Jit: 0.4.4
Mitsuba: 3.5.0

Steps to reproduce

1.here is my test function

class real_data_integrator(mi.SamplingIntegrator):
    def __init__(self, props):
        super().__init__(props)      
    def sample(self, 
               scene: mi.Scene, 
               sampler:mi.Sampler, 
               ray: mi.RayDifferential3f, 
               medium: mi.Medium = None,
               active: bool = True) -> tuple[mi.Color3f, bool, list[float]]:
        
        L = mi.Spectrum(0.0)

        # ---------------------- Direct emission ----------------------
        si = scene.ray_intersect(ray, active)        

        bsdf_ctx = mi.BSDFContext()
        sample1 = sampler.next_1d()
        sample2 = sampler.next_2d()
        bsdf: mi.BSDF = si.bsdf()

        bsdf_sample, bsdf_weight = bsdf.sample(
                    bsdf_ctx, si, sample1, sample2)

        self.shape_confirm(bsdf_weight)

        valid_ray = active & si.is_valid()
        active &= si.is_valid()
        
        cosThetaI = mi.Frame3f.cos_theta(si.wi)
        active &= (cosThetaI>0)        

        # Sample the emitter
        ds, em_weight = scene.sample_emitter_direction(
            si, sampler.next_2d(), True, active
        )
        active &= ds.pdf != 0.0
        
        self.shape_confirm(em_weight)
        return dr.select(True, L, 0.0), valid_ray, []
    
    @dr.wrap(source='drjit', target='torch')
    def shape_confirm(self,em_weight):
        print(em_weight.shape)

2.here is my test xml

<scene version="3.0.0">
    <default name="spp" value="32"/>
    <default name="res" value="512"/>
    <integrator type="My_integrator"/>

    <sensor type="perspective" id="sensor">
        <string name="fov_axis" value="smaller"/>
        <float name="near_clip" value="0.001"/>
        <float name="far_clip" value="100.0"/>
        <float name="focus_distance" value="1000"/>
        <float name="fov" value="40"/>
        <transform name="to_world">
            <lookat origin="0,  0,  3.6"
                    target="0,  0,  0"
                    up    ="0,  1,  0"/>
        </transform>
        <sampler type="independent">
            <integer name="sample_count" value="$spp"/>
        </sampler>
        <film type="hdrfilm">
            <integer name="width"  value="$res"/>
            <integer name="height" value="$res"/>
            <rfilter type="tent"/>
            <string name="pixel_format" value="rgb"/>
            <string name="component_format" value="float32"/>
        </film>
    </sensor>


<emitter type="directional">
    <vector name="direction" value="0.0, 0.0, -1.0"/>
    
    <!-- <spectrum name="irradiance" value="400:1.0 ,450:2.0 ,500:1.0 ,550:2.0 ,600:1.0 ,700:3.0"/> -->
    <rgb name="irradiance" value="1.0"/>
</emitter>

    <shape type="obj">
		<string name="filename" value="cube.obj" />
		<transform name="to_world">
            <scale value="1"/>
            <translate x="0" y="0" z="0"/>
        </transform>
        <bsdf type="diffuse">
        <spectrum name="reflectance" value="400:0.343, 404:0.445, 408:0.551, 412:0.624, 416:0.665, 420:0.687, 424:0.708, 428:0.723, 432:0.715, 436:0.71, 440:0.745, 444:0.758, 448:0.739, 452:0.767, 456:0.777, 460:0.765, 464:0.751, 468:0.745, 472:0.748, 476:0.729, 480:0.745, 484:0.757, 488:0.753, 492:0.75, 496:0.746, 500:0.747, 504:0.735, 508:0.732, 512:0.739, 516:0.734, 520:0.725, 524:0.721, 528:0.733, 532:0.725, 536:0.732, 540:0.743, 544:0.744, 548:0.748, 552:0.728, 556:0.716, 560:0.733, 564:0.726, 568:0.713, 572:0.74, 576:0.754, 580:0.764, 584:0.752, 588:0.736, 592:0.734, 596:0.741, 600:0.74, 604:0.732, 608:0.745, 612:0.755, 616:0.751, 620:0.744, 624:0.731, 628:0.733, 632:0.744, 636:0.731, 640:0.712, 644:0.708, 648:0.729, 652:0.73, 656:0.727, 660:0.707, 664:0.703, 668:0.729, 672:0.75, 676:0.76, 680:0.751, 684:0.739, 688:0.724, 692:0.73, 696:0.74, 700:0.737"/>
        </bsdf>
	</shape>

</scene>

[merlinND]

Hello @free7187,

Indeed, the number of wavelengths samples by each ray is a compile-time constant. By default, it is set to 4, e.g.:

mitsuba3/resources/mitsuba.conf.template

Line 329 in b70b7e3

"spectrum": "Spectrum<Float, 4>"

but you can in principle increase it in your mitsuba.conf file before compiling. I haven't tried this myself, so I can't guarantee that all possible values would work and / or remain fast.

The idea is that by sampling 4 wavelengths per ray and accumulating results from many rays, the full spectrum will still be correctly integrated.

[free7187]

Thank you for your answer @merlinND .
Correct a previous mistake, I set mi.set_variant('cuda_ad_spectral') not mi.set_variant('cuda_ad_rgb').

I tried configuring "spectrum": "Spectrum<Float, 31>"in the mitsuba.conf file and compile, but when I tried running it in Python, the debug threw an error:

Exception has occurred: ImportError
RuntimeError: Operation references type "Array31f", which lacks bindings

[merlinND]

I haven't tried this myself, but in principle it should be possible to add the missing bindings here:

https://github.com/mitsuba-renderer/drjit/blob/5bc1d4eed09fe05a5dc9d7eba9ca15ab946cf84b/src/python/bind.h#L610

[free7187]

Thank you again for your response.

However, after switching to the drjit branch you provided, I found that the drjit on this branch does not match the mitsuba3 I am currently using on the master branch.
For instance, the CMakeLists.txt in this branch does not include settings for drjit-extra, which leads to an error when generating the Mitsuba project with CMake:

CMake Error at ext/CMakeLists.txt:55 (set_target_properties):
set_target_properties Can not find target to add properties to: drjit-extra

[merlinND]

Hello @free7187,

In your first post, you mentioned that you are using Mitsuba: 3.5.0, which is why I linked to the corresponding DrJit version.
In the upcoming version, being developed on master, the bindings library has been switched from pybind11 to nanobind, but the idea of binding the missing array size remains the same.

That being said, master is under active development and you may encounter bugs.

[free7187]

Hello,

I have double-checked and apologize for the confusion regarding the version of Mitsuba I initially reported using. It appears that running python -m mitsuba.sys_info prioritizes displaying the Mitsuba environment installed via pip, whereas I am actually using a version compiled from source.

Returning to the issue at hand, to avoid encountering bugs, I recompiled Mitsuba: 3.5.0. This time, after configuring "spectrum": "Spectrum<Float, 31>" in the mitsuba.conf file, the error thrown by Python has changed:

Exception has occurred: AttributeError
pybind11::detail::get_type_info: unable to find type info for "drjit::Array<drjit::CUDAArray<float>,31>"

Thank you for your assistance.

[free7187]

Hello, @merlinND
I'd appreciate your insights on the latest question I posted. Thank you for your time!