UV to sensor back projection

[syllebra]

Hello, and congrats for this incredible work!
Mistuba is so easy to use that I tried to create synthetic datasets to train AI models.

For this, I need to "backproject" some point(s) defined in UV domain on a shape to the sensor domain (to create some image coordinates groundtruth data).

I found interface to get world point from a UV point using (python):
inter = bf.eval_parameterization(uv=mi.Point2f(u,v), ray_flags=mi.RayFlags.All, active=True)

I wonder if there exists sensor.from_world_p(pw) or something equivalent? I mainly use perspective projection and could recreate the perpective matrix maths, but it seemed that there must some code to do it, maybe in inverse rendering code?

Thank you very much.

[syllebra]

I managed to do it for perspective sensors (need width/height scaling:

pt = inter.p.numpy()[0]
prj = mi.perspective_projection(film.size(),film.crop_size(),film.crop_offset(),params["sensor.x_fov"],sensor.near_clip(),sensor.far_clip())
proj = (prj @ sensor.world_transform().inverse() @ inter.p)*480
proj = proj.numpy()[0][0:2]

Is it possible for other sensors types?

[njroussel]

Hi @syllebra

I think you're looking for this method: Sensor.sample_direction().

It won't do a visibility test for you, if you desire that. I think the closest example we have to something that uses this is the ptracer's conect_sensor method. In short, this method is also trying to connect an arbitrary point in the scene to a specific sensor.