disable speed perturbation by default

[JinZr]

No description provided.

[danpovey]

We should probably do this, even though it may make some of the recipes not fully reproducible (i.e. it would require to change the learning rate schedules and num-epochs).
It might make sense to actually change the LR schedules and epochs at the same time. This would not be fully
foolproof, since some people will have previously-dumped non-perturbed features on disk. But it will ensure that
from a fresh checkout things would be basically reproducible.
(if README files have example usages, we could perhaps just write the alternative command lines that you would use
if the features were not dumped with speed perturbation, with the num-epochs and if applicable lr-epochs multiplied by 3.)

[pzelasko]

It doesn’t help anymore?

[JinZr]

Not really. The thing is when the amount of data reaches a certain point, speed perturbation stops bringing performance improvement to the system and would be a waste of space and time. So we figure that we can leave a switch and let users decide whether they want it or not.

…

On Thu, Jul 20, 2023 at 8:39 PM Piotr Żelasko ***@***.***> wrote: It doesn’t help anymore? — Reply to this email directly, view it on GitHub <#1176 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOON42GPJHMYBJCV2X7Y6HTXRERHNANCNFSM6AAAAAA2RCZYGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[pkufool]

It doesn’t help anymore?

We did experiments on librispeech dataset, the performance of speed-pertur is the same as no speed-pertur (3 times epochs).

[yfyeung]

We did experiments on librispeech dataset, the performance of speed-perturb is the same as no speed-perturb (3 times epochs).

Somehow slightly worse than with perturb

[yfyeung]

Maybe you can use:

if speed_perturb and "train" in partition:

[JinZr]

applied requested changes to the latest commit

…

On Sun, Jul 23, 2023 at 6:38 PM Yifan Yang ***@***.***> wrote: ***@***.**** requested changes on this pull request. Maybe you can use: if speed_perturb and "train" in partition: — Reply to this email directly, view it on GitHub <#1176 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOON42FKI56BULFBYYARSUDXRT5K7ANCNFSM6AAAAAA2RCZYGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[JinZr]

I suppose we could do this: to incorporate with the new `speed-perturb` arg, we could set the default value of this arg to False, but give it a True value in the `prepare.sh` script of respective recipes. Starters could easily reproduce the results by running `prepare.sh` without us modifying `num-epochs` and `lr-epochs` in the training scripts of individual models. But still, this solution is not fully foolproof.

…

On Thu, Jul 20, 2023 at 5:29 PM Daniel Povey ***@***.***> wrote: We should probably do this, even though it may make some of the recipes not fully reproducible (i.e. it would require to change the learning rate schedules and num-epochs). It might make sense to actually change the LR schedules and epochs at the same time. This would not be fully foolproof, since some people will have previously-dumped non-perturbed features on disk. But it will ensure that from a fresh checkout things would be basically reproducible. (if README files have example usages, we could perhaps just write the alternative command lines that you would use if the features were not dumped with speed perturbation, with the num-epochs and if applicable lr-epochs multiplied by 3.) — Reply to this email directly, view it on GitHub <#1176 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOON42G2LREQT4OS2A4OKK3XRD27DANCNFSM6AAAAAA2RCZYGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[danpovey]

There is already a PR (sorry IDK the PR#) to change the arg to false in the prepare.sh scripts.

…

On Thu, Jul 27, 2023 at 5:43 PM zr_jin ***@***.***> wrote: I suppose we could do this: to incorporate with the new `speed-perturb` arg, we could set the default value of this arg to False, but give it a True value in the `prepare.sh` script of respective recipes. Starters could easily reproduce the results by running `prepare.sh` without us modifying `num-epochs` and `lr-epochs` in the training scripts of individual models. But still, this solution is not fully foolproof. On Thu, Jul 20, 2023 at 5:29 PM Daniel Povey ***@***.***> wrote: > We should probably do this, even though it may make some of the recipes > not fully reproducible (i.e. it would require to change the learning rate > schedules and num-epochs). > It might make sense to actually change the LR schedules and epochs at the > same time. This would not be fully > foolproof, since some people will have previously-dumped non-perturbed > features on disk. But it will ensure that > from a fresh checkout things would be basically reproducible. > (if README files have example usages, we could perhaps just write the > alternative command lines that you would use > if the features were not dumped with speed perturbation, with the > num-epochs and if applicable lr-epochs multiplied by 3.) > > — > Reply to this email directly, view it on GitHub > <#1176 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AOON42G2LREQT4OS2A4OKK3XRD27DANCNFSM6AAAAAA2RCZYGE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#1176 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO5U2PKSNARHMSRDFQTXSIZ5LANCNFSM6AAAAAA2RCZYGE> . You are receiving this because you commented.Message ID: ***@***.***>

1

[csukuangfj]

Thanks! Left a minor comment. Otherwise, it looks good to me.

[csukuangfj]

Thanks!