You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When enabled, --noise_offset increases the value range of a lora. Example, 500 steps, teaching a lora a dataset containing a healthy dark background mix:
That's a noise_offset of 0.1, prompting for "black background". Without it the lora can't even learn or keep existing extremes. The obvious issue with it is that when trained with offset the output got incredibly mushier. Training more helps with quality, but it's still poor. It'll pick a lot of deformations and inferior generic shading and colors, and worse, in later epochs the noise offset effect on range fades, without texture definition ever catching up. After 1,000 steps:
--noise_offset_random_strength doesn't seem to really help. I've had no luck with lowering the noise offset either, it only makes the lora forget the increased value range sooner.
I've been experimenting with splicing block weights from undertrained noise_offset loras with overtrained no noise equivalents. Looks better than a noise_offset lora by itself, but is still very underwhelming.
I wonder if anyone else already messed with it and came up a good training parameter mix to stave off the decrease in quality and keep it working for longer.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
When enabled,
--noise_offset
increases the value range of a lora. Example, 500 steps, teaching a lora a dataset containing a healthy dark background mix:That's a noise_offset of 0.1, prompting for "black background". Without it the lora can't even learn or keep existing extremes. The obvious issue with it is that when trained with offset the output got incredibly mushier. Training more helps with quality, but it's still poor. It'll pick a lot of deformations and inferior generic shading and colors, and worse, in later epochs the noise offset effect on range fades, without texture definition ever catching up. After 1,000 steps:
--noise_offset_random_strength
doesn't seem to really help. I've had no luck with lowering the noise offset either, it only makes the lora forget the increased value range sooner.I've been experimenting with splicing block weights from undertrained noise_offset loras with overtrained no noise equivalents. Looks better than a noise_offset lora by itself, but is still very underwhelming.
I wonder if anyone else already messed with it and came up a good training parameter mix to stave off the decrease in quality and keep it working for longer.
Beta Was this translation helpful? Give feedback.
All reactions