NOT AN ISSUE: Fixed colab notebook to work as of October 14th, 2021

[qwertyuu]

Hello!

Thanks a whole for implementing this, it works wonders.

There has been some time since the notebook was written, and it did not work out of the box anymore. So I went ahead and made this one:
https://colab.research.google.com/drive/1EPoNkTHIyYeLq_fZZY5HS1ecjCUQEGvg?usp=sharing

It is still running on my end but it did successfully generate the first utterance. It's very very nice. Thanks again!

[vaibhawkhemka]

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

• https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
• https://github.com/tensorflow/addons
• https://github.com/tensorflow/io (for I/O related ops)
  If you depend on functionality not listed there, please file an issue.

loaded model at logs-Tacotron/pretrained/model.ckpt-189500
Hyperparameters:
allow_clipping_in_normalization: True
attention_dim: 128
attention_filters: 32
attention_kernel: (31,)
cleaners: english_cleaners
cumulative_weights: True
decoder_layers: 2
decoder_lstm_units: 1024
embedding_dim: 512
enc_conv_channels: 512
enc_conv_kernel_size: (5,)
enc_conv_num_layers: 3
encoder_lstm_units: 256
fft_size: 1024
fmax: 7600
fmin: 125
frame_shift_ms: None
griffin_lim_iters: 60
hop_size: 256
impute_finished: False
input_type: raw
log_scale_min: -32.23619130191664
mask_encoder: False
mask_finished: False
max_abs_value: 4.0
max_iters: 2500
min_level_db: -100
num_freq: 513
num_mels: 80
outputs_per_step: 1
postnet_channels: 512
postnet_kernel_size: (5,)
postnet_num_layers: 5
power: 1.1
predict_linear: False
prenet_layers: [256, 256]
quantize_channels: 65536
ref_level_db: 20
rescale: True
rescaling_max: 0.999
sample_rate: 22050
signal_normalization: True
silence_threshold: 2
smoothing: False
stop_at_any: True
symmetric_mels: False
tacotron_adam_beta1: 0.9
tacotron_adam_beta2: 0.999
tacotron_adam_epsilon: 1e-06
tacotron_batch_size: 16
tacotron_decay_learning_rate: True
tacotron_decay_rate: 0.4
tacotron_decay_steps: 50000
tacotron_dropout_rate: 0.5
tacotron_final_learning_rate: 1e-05
tacotron_initial_learning_rate: 0.001
tacotron_reg_weight: 1e-06
tacotron_scale_regularization: True
tacotron_start_decay: 50000
tacotron_teacher_forcing_ratio: 1.0
tacotron_zoneout_rate: 0.1
trim_silence: True
use_lws: True
Constructing model: Tacotron
WARNING:tensorflow:From /root/Tacotron-2/tacotron/synthesizer.py:16: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/synthesizer.py:19: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/tacotron.py:52: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/zoneout_LSTM.py:88: The name tf.nn.rnn_cell.LSTMStateTuple is deprecated. Please use tf.compat.v1.nn.rnn_cell.LSTMStateTuple instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:16: conv1d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.keras.layers.Conv1D instead.
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/layers/convolutional.py:218: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use layer.__call__ method instead.
WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:17: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.BatchNormalization instead. In particular, tf.control_dependencies(tf.GraphKeys.UPDATE_OPS) should not be used (consult the tf.keras.layers.batch_normalization documentation).
WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:20: dropout (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dropout instead.
WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:82: bidirectional_dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use keras.layers.Bidirectional(keras.layers.RNN(cell)), which is equivalent to this API
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/rnn.py:464: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use keras.layers.RNN(cell), which is equivalent to this API
WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/zoneout_LSTM.py:145: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/rnn.py:244: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/attention.py:161: The name tf.layers.Conv1D is deprecated. Please use tf.compat.v1.layers.Conv1D instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/attention.py:164: The name tf.layers.Dense is deprecated. Please use tf.compat.v1.layers.Dense instead.

WARNING:tensorflow:From /root/Tacotron-2/tacotron/models/modules.py:146: MultiRNNCell.init (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.StackedRNNCells, and will be replaced by that in Tensorflow 2.0.
Traceback (most recent call last):
File "synthesize.py", line 34, in
main()
File "synthesize.py", line 28, in main
tacotron_synthesize(args)
File "/root/Tacotron-2/tacotron/synthesize.py", line 81, in tacotron_synthesize
run_eval(args, checkpoint_path, output_dir, sentences)
File "/root/Tacotron-2/tacotron/synthesize.py", line 14, in run_eval
synth.load(checkpoint_path)
File "/root/Tacotron-2/tacotron/synthesizer.py", line 24, in load
self.model.initialize(inputs, input_lengths)
File "/root/Tacotron-2/tacotron/models/tacotron.py", line 112, in initialize
CustomDecoder(decoder_cell, self.helper, decoder_init_state),
File "/root/Tacotron-2/tacotron/models/custom_decoder.py", line 47, in init
if not rnn_cell_impl._like_rnncell(cell): # pylint: disable=protected-access
AttributeError: module 'tensorflow.python.ops.rnn_cell_impl' has no attribute '_like_rnncell'

@qwertyuu still giving the above issue after running the notebook.

Above issue produced after executing this cell. Waiting for your help...

[qwertyuu]

On it

[qwertyuu]

It is now generating on my end. Can you validate it also works on yours @vaibhawkhemka ?

Thanks for testing :)

[vaibhawkhemka]

@qwertyuu It works!! Great Job!!
Btw Curious to know what changes u did??

[qwertyuu]

@vaibhawkhemka I changed 2 things:

• The moment I switch the environment from TF 2.x to TF 1.x (seems like there is a timing issue with the dependencies, very weird, did not go in the details as of why)
• I forked the Tacotron-2 project and removed the check that used _like_rnncell (the error that you had). Luckily this was just an assertion that could be removed.

Nothing else seemed to cause compatibility problems so it worked. Have fun!

[LottieSebes]

Hi @qwertyuu
I was trying to replicate this book on my colab account but at the stage of installing the dependencies I got the following error. It it let me proceed to the next cells so I did. Here is the first error:

---

ERROR: Could not find a version that satisfies the requirement tensorflow<=1.9.0 (from versions: 1.13.1, 1.13.2, 1.14.0, 1.15.0, 1.15.2, 1.15.3, 1.15.4, 1.15.5, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0, 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.4.0rc0, 2.4.0rc1, 2.4.0rc2, 2.4.0rc3, 2.4.0rc4, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.7.1, 2.8.0rc0, 2.8.0rc1, 2.8.0)
ERROR: No matching distribution found for tensorflow<=1.9.0
|████████████████████████████████| 312 kB 33.8 MB/s
|████████████████████████████████| 50 kB 8.2 MB/s
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.8.0 requires tf-estimator-nightly==2.8.0.dev2021122109, which is not installed.
tensorflow 2.8.0 requires keras<2.9,>=2.8.0rc0, but you have keras 2.2.4 which is incompatible.
|████████████████████████████████| 13.8 MB 30.3 MB/s
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.8.0 requires tf-estimator-nightly==2.8.0.dev2021122109, which is not installed.
yellowbrick 1.4 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible.
xarray 0.18.2 requires numpy>=1.17, but you have numpy 1.15.4 which is incompatible.
tensorflow 2.8.0 requires keras<2.9,>=2.8.0rc0, but you have keras 2.2.4 which is incompatible.
tensorflow 2.8.0 requires numpy>=1.20, but you have numpy 1.15.4 which is incompatible.
tables 3.7.0 requires numpy>=1.19.0, but you have numpy 1.15.4 which is incompatible.
scikit-image 0.18.3 requires numpy>=1.16.5, but you have numpy 1.15.4 which is incompatible.
pywavelets 1.2.0 requires numpy>=1.17.3, but you have numpy 1.15.4 which is incompatible.
pyerfa 2.0.0.1 requires numpy>=1.17, but you have numpy 1.15.4 which is incompatible.
pyarrow 6.0.1 requires numpy>=1.16.6, but you have numpy 1.15.4 which is incompatible.
plotnine 0.6.0 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible.
pandas 1.3.5 requires numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you have numpy 1.15.4 which is incompatible.
kapre 0.3.7 requires numpy>=1.18.5, but you have numpy 1.15.4 which is incompatible.
jaxlib 0.3.0+cuda11.cudnn805 requires numpy>=1.19, but you have numpy 1.15.4 which is incompatible.
jax 0.3.1 requires numpy>=1.19, but you have numpy 1.15.4 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
cupy-cuda111 9.4.0 requires numpy<1.24,>=1.17, but you have numpy 1.15.4 which is incompatible.
astropy 4.3.1 requires numpy>=1.17, but you have numpy 1.15.4 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
|████████████████████████████████| 421 kB 33.0 MB/s
Building wheel for pysptk (setup.py) ... done
|████████████████████████████████| 131 kB 30.3 MB/s
|████████████████████████████████| 58 kB 7.2 MB/s
|████████████████████████████████| 1.5 MB 37.3 MB/s
|████████████████████████████████| 53.9 MB 228 kB/s
|████████████████████████████████| 228 kB 53.6 MB/s
Building wheel for librosa (setup.py) ... done
Building wheel for matplotlib (setup.py) ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
yellowbrick 1.4 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible.
seaborn 0.11.2 requires matplotlib>=2.2, but you have matplotlib 2.0.2 which is incompatible.
scikit-image 0.18.3 requires numpy>=1.16.5, but you have numpy 1.15.4 which is incompatible.
pycocotools 2.0.4 requires matplotlib>=2.1.0, but you have matplotlib 2.0.2 which is incompatible.
plotnine 0.6.0 requires matplotlib>=3.1.1, but you have matplotlib 2.0.2 which is incompatible.
plotnine 0.6.0 requires numpy>=1.16.0, but you have numpy 1.15.4 which is incompatible.
mizani 0.6.0 requires matplotlib>=3.1.1, but you have matplotlib 2.0.2 which is incompatible.
kapre 0.3.7 requires librosa>=0.7.2, but you have librosa 0.5.1 which is incompatible.
kapre 0.3.7 requires numpy>=1.18.5, but you have numpy 1.15.4 which is incompatible.
arviz 0.11.4 requires matplotlib>=3.0, but you have matplotlib 2.0.2 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
|████████████████████████████████| 125 kB 28.0 MB/s
|████████████████████████████████| 2.0 MB 68.2 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
|████████████████████████████████| 150 kB 66.0 MB/s
|████████████████████████████████| 410 kB 67.6 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Building wheel for nnmnkwii (PEP 517) ... done
Building wheel for pysptk (PEP 517) ... done
Building wheel for lws (setup.py) ... done

---

The next error happened at the 'Mel-spectrogram prediction by Tacoron2' at which point I got the following stacktrace but it still let me continue:

---

Traceback (most recent call last):
File "synthesize.py", line 2, in
from tacotron.synthesize import tacotron_synthesize
File "/root/Tacotron-2/tacotron/synthesize.py", line 4, in
from hparams import hparams, hparams_debug_string
File "/root/Tacotron-2/hparams.py", line 1, in
import tensorflow as tf
File "/tensorflow-1.15.2/python3.7/tensorflow/init.py", line 99, in
from tensorflow_core import *
File "/tensorflow-1.15.2/python3.7/tensorflow_core/init.py", line 36, in
from tensorflow._api.v1 import compat
File "/tensorflow-1.15.2/python3.7/tensorflow_core/_api/v1/compat/init.py", line 23, in
from tensorflow._api.v1.compat import v1
File "/tensorflow-1.15.2/python3.7/tensorflow_core/_api/v1/compat/v1/init.py", line 672, in
from tensorflow_estimator.python.estimator.api._v1 import estimator
File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/init.py", line 10, in
from tensorflow_estimator._api.v1 import estimator
File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/_api/v1/estimator/init.py", line 12, in
from tensorflow_estimator._api.v1.estimator import inputs
File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/_api/v1/estimator/inputs/init.py", line 10, in
from tensorflow_estimator.python.estimator.inputs.numpy_io import numpy_input_fn
File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/python/estimator/inputs/numpy_io.py", line 26, in
from tensorflow_estimator.python.estimator.inputs.queues import feeding_functions
File "/tensorflow-1.15.2/python3.7/tensorflow_estimator/python/estimator/inputs/queues/feeding_functions.py", line 40, in
import pandas as pd
File "/usr/local/lib/python3.7/dist-packages/pandas/init.py", line 22, in
from pandas.compat import (
File "/usr/local/lib/python3.7/dist-packages/pandas/compat/init.py", line 15, in
from pandas.compat.numpy import (
File "/usr/local/lib/python3.7/dist-packages/pandas/compat/numpy/init.py", line 7, in
from pandas.util.version import Version
File "/usr/local/lib/python3.7/dist-packages/pandas/util/init.py", line 1, in
from pandas.util._decorators import ( # noqa
File "/usr/local/lib/python3.7/dist-packages/pandas/util/_decorators.py", line 14, in
from pandas._libs.properties import cache_readonly # noqa
File "/usr/local/lib/python3.7/dist-packages/pandas/_libs/init.py", line 13, in
from pandas._libs.interval import Interval
File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

---

Then the Waveform synthesis by Wavenet stage failed in the third cell because tacotron_output folder did not exist:

---

`FileNotFoundError Traceback (most recent call last)
in ()
2 from tqdm import tqdm
3
----> 4 with open("../Tacotron-2/tacotron_output/eval/map.txt") as f:
5 maps = f.readlines()
6 maps = list(map(lambda x:x[:-1].split("|"), maps))

FileNotFoundError: [Errno 2] No such file or directory: '../Tacotron-2/tacotron_output/eval/map.txt'`

---

I suppose the problem lies with the dependencies. Do you have any ideas about how to resolve this? Thanks!