How to add new diffusion model pipeline? Like kandinsky-2-2 for example

[Pashtetickus]

Hi!
I'm curios about how to add a new diffusion pipeline from diffusers.
I'm interested in kandinsky.
My question is what I should be looking at as a reference - what modules shoud be there and how to implement them? How to parse any new pipeline on parts that threestudio needs?

[bennyguo]

Hi! You may want to take a look at threestudio/models/guidance/stable_diffusion_guidance.py. Basically you need to compute the loss for an input image, which is often done by evaluating the UNet (and VAE) of the diffusion model, but the specific operation varies from pipeline to pipeline. By the way, I've been re-writing all the guidance recently and you should feel more easy to implement new guidance with the refactored code.

[Pashtetickus]

Great! I'll try to figure it out and hopefully it will be a new branch with kandinsky. Thank you