Does this code mess up clip_grad_norm when training text encoder?

[araleza]

Hey, just came back to kohya-ss code after a while, and I saw this in sdxl_train.py:

accelerator.backward(loss)
if accelerator.sync_gradients and args.max_grad_norm != 0.0:
    params_to_clip = []
    for m in training_models:
        params_to_clip.extend(m.parameters())
    accelerator.clip_grad_norm_(params_to_clip, args.max_grad_norm)

optimizer.step()

It looks like all parameters for the three models (tenc1, tenc2, unet) are having a singular gradient normal formed for them, which is then clipped. Shouldn't each of the three models get its own normal generated, and these could be clipped instead? Might lead to better training.