Training Curves

[atosystem]

Hi!
I'm wondering if anyone can provide some training statistics (e.g. the tensorboard data during 1st and 2nd stage training)?
I want to make sure if I'm on the right track.

Thanks!

[martinambrus]

I can provide tensorboard data of stage 1 training (150 epochs) + the stage 2 training (34 epochs so far) + train log. The model I'm training already sounds okay but I expect much improvement once joint training kicks in.

Also, please bear in mind that I was resuming training multiple times, so I guess some of those tensorboard data might reflect that (I've not used tensorboard before, so I don't know whether it rewrites old data with new training or appends to them... however, the first 150 epochs of stage 1 seem to be there).

One last thing - this is training data from training a single-speaker English model from scratch.

https://www.dropbox.com/scl/fi/facz5nmqy38g0mszlq0fz/tensorboard.zip?rlkey=efk8lxo96pul86i5a9uuhdyg4&st=qpi7klbe&dl=1

[Karesto]

Hi, the link of your file seems dead, is it possible for you to reupload them ?
Thanks.

[martinambrus]

Hi, the link of your file seems dead, is it possible for you to reupload them ? Thanks.

Thanks for letting me know. I've uploaded it to Dropbox and changed the URL in my comment above.