Audio preprocessing

[JRMeyer]

>>> CrazyJoeDevola
[September 6, 2020, 9:37am]

Hi

Does anyone have experience on how much consistent normalized audio have
on the final output? slash
What level are you normalizing at? - 6db?

[This is an archived TTS discussion thread from discourse.mozilla.org/t/audio-preprocessing]

[JRMeyer]

>>> dkreutz
[September 6, 2020, 5:44pm]

You can configure TTS/Tacotron to do preprocessing and normalize audio,
so there is no need to do that yourself. But what is helful is to have
consistent average loudness. I preprocessed our dataset with
RNNoise and
pyloudnorm (latter with an
average loudness of -25dB, at bit lower than usual to avoid clipping
artefacts).

[Archived Post]

[JRMeyer]

>>> CrazyJoeDevola
[September 7, 2020, 6:47am]

> th an average loudness of -25dB, at bit lower than usual to avoid
> clipping ar

Thanks. That probably makes it easier. I use desktop software (Audacity)
to process all files right now.

What about other noise and background rumble in the audio, to what
extent will that affect the final TTS?

[Archived Post]

[JRMeyer]

>>> dkreutz
[September 7, 2020, 7:02am]

> What about other noise and background rumble in the audio, to what
> extent will that affect the final TTS?

Audio signal should be as clean as possible. TTS model can not
distinguish between speech signal and unwanted background noise and will
learn e.g. room echo noise as part of the signal otherwise - which leads
to echoing and/or robotic voice sound.

As already mentioned RNNoise removes a lot of noise very effectively. In
addition you can do lowpass (e.g. 8000Hz) and highpass (e.g. 50Hz)
filtering and set fmin/fmax parameters in config file.

[Archived Post]

[JRMeyer]

>>> CrazyJoeDevola
[September 9, 2020, 10:44am]

> As already mentioned RNNoise removes a lot of noise very effectively.
> In addition you can do lowpass (e.g. 8000Hz) and highpass (e.g. 50Hz)
> filtering and set fmin/fmax parameters in config file.

Thanks a lot. Where can i read more about this? I suppose this is
RNNoise? https://github.com/xiph/rnnoise

Are the other variables entered into the config?

> (e.g. 8000Hz) and highpass (e.g. 50Hz) filtering and set fmin/fmax

Do they need to be set during the whole training or just used when
syntheseizing? slash
Or can i change them later once the model has been trained for a while?

[Archived Post]

[JRMeyer]

>>> CrazyJoeDevola
[September 9, 2020, 10:46am]

Also, sometimes i get a long 'noise trail' when rendering audio using
either TTS test audios or together with Pwgan decoder.

I wanna apply the best possible vocoder (doesnt matter if it takes
longer time). Which would your recommend?

[Archived Post]

[JRMeyer]

>>> dkreutz
[September 9, 2020, 1:00pm]

> I suppose this is RNNoise? https://github.com/xiph/rnnoise

Yes, that is the correct one.

> Do they need to be set during the whole training or just used when
> syntheseizing?

fmin/fmax are parameters for TTS internal audio-processor and must be
set at training time, they don't have effect at inference time. slash
As you want to train a vocoder model make sure to use the same audio
process settings as for the training of the Taco/TTS model. If I
remember correctly the dev-branch already uses same config for Taco and
Vocoder training.

> I wanna apply the best possible vocoder (doesnt matter if it takes
> longer time). Which would your recommend?

I don't have too much practical experience with vocoders yet, but to my
understanding best results can be achieved with ParallelWaveGAN and/or
Multiband-MelGAN.

[Archived Post]

[taras-sereda]

Thanks for sharing details on audio normalization and denoising!
I'm curious if you have any comparison of model performance on loud-normalized audio vs not normalized?

Hypothesis:
pyludnorm uses not very sophisticated loudness normalization algorithm, so a model can potentially learn necessary filters to perform renormalization.

The same question is on the performance comparison of models trained on denoised vs not denoised data.

The statements posted above make sense to me, that training models on normalized and denoised data is easier than on noisy data. Though on the other hand abundance of noisy speech available on the web, makes it very appealing to use it for TTS models.

[erogol]

If denoiser is good, then denoised data is always better.

Loud normalization also helps. I find just min-max norm and * 0.9 works as good as any advance loud-normalization