Question about regularization images

[smolkat64]

Whenever I train LoRA network with regularization images I always get the same result: my network learns almost nothing, generated images with this network look only slightly similar to the dataset images. But results are fine when I don't use them. Any tips? For example I prepared 59 images of "1girl" generated in webui for a dataset of 59 images of an anime character that have caption files like "<tok>,1girl, short hair...". Am I doing it right?

[mgz-dev]

These observations are specific to LoRA since I haven't tested or reviewed the code to confirm what happens during other forms of training with regularization:

When you train with regularization enabled, for every training image (repeats are counted as separate training images) a regularization image gets added into your dataset.

This means, for example, if you had 10 training images with regularization enabled, your dataset total size is now 20 images. If you trained with 10 images and 10 repeats, you now have 200 images (with 100 regularization images).

Because your dataset has been inflated with regularization images, you would need to have twice the number of steps to see your original training images the same number of times as without regularization.

Additionally, if you are batching during training, your loss per step is now lower (because your regularization images will have lower loss than your training images bringing the average down) making training slower. This will need to be compensated for by either increasing step count or increasing LR relative to training without regularization.

If you are training a character and using a unique activation token, the captions of your regularization images should not include the unique activation token.