differentiable rendering with SMPL-X

[ruoshiliu]

Hi,

This is a very general question and I'm relatively new to the field so I'm sorry if my question doesn't make sense at all. I'm trying to implement a framework where I'd like to perform optimization in the latent space of SMPL-X human model for differentiable rendering. In other words, in the scene I'm constructing, the optimizer can adjust the latent variable of SMPL-X, which is mapped to surface meshes of human by a neural network, in order to optimize for some photometric loss. Since SMPL-X is implemented in PyTorch, would it be easier and stabler to work with mitsuba-2 or mitsuba-3? Also, what might be some necessary resources I can go through to get a better idea of how to implement this framework? I came across this page in mistuba2 documentaion which seems quite useful: https://mitsuba2.readthedocs.io/en/latest/src/inverse_rendering/pytorch.html.

Thanks,

[Speierers]

Hi @ruoshiliu,

This all sounds very interesting. I would suggest you take a look at this new tutorial which is about using Mitsuba 3 in a Pytorch pipeline. This should be helpful to get you started with your project.