Ask for some help to estimate BRDF and mesh from real world textured object?

[Start1er]

In real world multi-view textured object data. Several works were illustrated the mesh and BRDF estimation examples, but I try to do it with my photos. Unfortunately, even I use regulization of mesh vertexes like https://github.com/rgl-epfl/large-steps-pytorch does, it's hard to recover the correct geometry. So I guess the following the chanlenge I might not do it correctly. I may need some help with it:

1. How to initial a rough mesh as start? Colmap calibration and then?
2. How to make sure the initial mesh has the same topology as final estimated one?
3. Does real world textured object BRDF and shape from multi-view images using PBDR still a very challenge problem to be solved in the furture?
   I will appreatiate it if someone could help me make it clear!

[njroussel]

Hi @Start1er

Reconstructions from real world captures come with a significant number of new challenges, as you've seen. I don't know enough about your setup, but unless you don't know exactly every parameter of your scene, you'll have a hard time getting decent results with PBR methods. Namely, you'll want to accurately model your light source and sensor if you wish to recover the BRDF.

The unknown topology issue is pretty common, I'd recommend circumventing that by using an SDF optimization and then running through some meshing algorithm (marching cubes for example). There's not much more that I can offer currently.

[Start1er]

Hi @njroussel :
Thanks for your kindly suggestion and reply, and I agree with your point, I think recover PBR material and shape must utilize a well-calibrated cameras poses and intrinsics, meanwhile, a known light source such as a co-located point light with camera like https://luanfujun.com/InverseMeshSVBRDF/ or a well-known spot light like in https://rgl.epfl.ch/publications/Loubet2019Reparameterizing is needed Then both of these works use a initial mesh to start crose-to-fine optimization for vertexes position and texture. And I struggle without the more optimization detail just using a kind of general-purposed renderer e.g. mitsuba. I wish in the further, I can easily implement this task in mitsuba!