Several examples doesn't work (Sashimi checkpoints / sampleRNN training)

[RochMollero]

Hello,

So I've cloned the repo and downloaded the sashimi checkpoints on hugging face in order to reproduce them and apply the method on my dataset. I've found three main problems:

any audio generated with sashimi is basically noise i.e. :

this command :
python -m generate experiment=audio/sashimi-youtubemix checkpoint_path=checkpoints/sashimi_youtubemix_v4.pt n_samples=10 l_sample=16000 load_data=false

leads to :
sashimi generation.zip

=> It does work with SampleRNN though.

=> With Wavenet it doesn't work either python -m generate experiment=audio/wavenet-sc09 checkpoint_path=checkpoints/wavenet_sc09.pt n_samples=1 l_sample=16000 load_data=false gives the error :

module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for SequenceLightningModule:
	Missing key(s) in state_dict: "encoder.0.weight", "decoder.0.output_transform.weight", "decoder.0.output_transform.bias". 
	Unexpected key(s) in state_dict: "encoder.weight", "decoder.output_transform.weight", "decoder.output_transform.bias". 

Training with SampleRNN doesn't work on youtubemix and beethoven:

so first i've noticed there was a reorganisation probably so I had to modify some .yaml for the new architecture

+++ b/configs/experiment/audio/samplernn-youtubemix.yaml
@@ -6,7 +6,7 @@ defaults:
   - /task: multiclass_classification
   - /optimizer: adamw
   - /scheduler: plateau
-  - /model: samplernn
+  - /model: baseline/samplernn

but then it still don't work, with
python -m train experiment=audio/samplernn
i get

    assert self.val_sanity_progress_bar_id is not None
AssertionError

and if I deactivate RichProgressBar, then I can see that it's because val/loss is not logged and I absolutely can't understand why. I've tried hunting differences between youtubemix and sc09 onwhich the training starts, but to no success.

What happens ? I've seen the "kernel what changed" to V4: is this why examples on sashmi are not working ? Should I go back to v3 ? I use pykeops but not the extension/kernels, would that be a reason ?

Regards,

Edit : also python setup.py install is deprectaed, I'm not sure how i should install the new kernels

[domdal]

Hello,

So I've cloned the repo and downloaded the sashimi checkpoints on hugging face in order to reproduce them and apply the method on my dataset. I've found three main problems:

any audio generated with sashimi is basically noise i.e. :

this command : python -m generate experiment=audio/sashimi-youtubemix checkpoint_path=checkpoints/sashimi_youtubemix_v4.pt n_samples=10 l_sample=16000 load_data=false

leads to : sashimi generation.zip

I have the same problem that ever audio generate with sashimi_sc09_v4.pt and sashimi_youtubemix_v4.pt is only noise.
The pretrained networks were downloaded at https://huggingface.co/krandiash/sashimi-release

I also tried to train my own network -- audio/sashimi-sc09, but it is also only noise

[albertfgu]

Sorry for the inconvenience. If you've trained your own model from scratch and it's still noise, there is probably a newly introduced mistake in the stepping code.

Actually, I do remember this issue but I thought it was fixed. Check this line here:

s4/src/models/sequence/modules/s4block.py

Line 170 in a246043

y = self.activation(y)

There was missing a corresponding activation applied in the step function:

s4/src/models/sequence/modules/s4block.py

Line 199 in a246043

if self.gate is not None:

Hopefully fixing that will fix the generation issues.

[domdal]

Thank you for the fast reply. It is working now

[albertfgu]

Fixed by 35cc242