Add image as control condition

[GG-Bonds]

I tried adding images as more control conditions, but our model starts oscillating after the second epoch and does not decrease. Has the author done this experiment before?

[nuneslu]

Hi! Could you share a picture with the training loss? I assume that by “starts oscilating and does not decrease” you mean the loss right?

[GG-Bonds]

Thank you for your response!
you are right. Below is a screenshot of the model loss. (It should be noted that I used 8 GPUs for training, and other parameters have not changed)

[nuneslu]

Hi, the loss for diffusion is in general quite unstable, usually quickly converges and doesn’t change much (see this issue). So I would suggest you to the model training for a few epochs and check the results during the intermediary checkpoints. If the results are not good you can share it here or via email with me and I will be more than happy to discuss it and try helping you make it work conditioned to images as well. :)

[RoboticsZhang]

Hey, I wonder that how did you add images as control condition in this task? Could you briefly introduce your methods?