-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
problem while training #15
Comments
the argument Unless you specifically want to reproduce our WMT16 systems, I recommend that you have a look at https://github.com/EdinburghNLP/wmt17-scripts/tree/master/training , which uses better hyperparameters, or even https://github.com/EdinburghNLP/wmt17-transformer-scripts , which shows how to train a Transformer with Nematus. |
Thank you very much for your reply. I want to train FACTORED NEURAL MACHINE TRANSLATION with POS of src, I wanna to know whether I can train it in the two new ways you mentioned above or not. |
yes, this is possible. You can do preprocessing as instructed in this repository (or with your own modifications). To then train the actual model, I suggest your start with this config https://github.com/EdinburghNLP/wmt17-transformer-scripts/blob/master/training/scripts/train.sh and make some additions to define the number of factors and the embedding size per factor (for example |
I am trying https://github.com/EdinburghNLP/wmt17-transformer-scripts with sample files, but while running training/scripts/train.sh happend: INFO: Initializing model parameters from scratch... |
can you show the first line of your (source-side) training data, and your training arguments? It is likely that the number of actual factors your data contains is different from the number you specified as training argument. |
corpus.bpe.en:
..... training/scripts/train.sh:
errors :
I only changed the batch_size to 64 as I have one GPU . |
Now I use my own corpus to train https://github.com/EdinburghNLP/wmt17-transformer-scripts , but: really sorry for so many questions ... |
ok, I neglected to check, but factors aren't currently implemented for the transformer architecture. You can still look at https://github.com/EdinburghNLP/wmt17-scripts/tree/master/training to see how to use the RNN-version of Nematus with the command-line interface, and use the hyperparameters from there as a starting point. For your previous entry, I'm a bit confused because you say that you want to use factored models, but your input doesn't contain any extra factors. It is your own responsibility to preprocess the data pass it to Nematus in the right format: https://github.com/EdinburghNLP/nematus/blob/master/doc/factored_neural_machine_translation.md As to your batch size, training with small batches will actually cost you quite a bit of translation quality. You can use |
Sorry I didn't make myself clear. At the begining I want to use data from
Thanks, it's really help me. I added
Thanks, and I have started trying wmt17-scripts. Now It starts taining but stopped with these mesage:
my config:
My src corpus:
|
./train.sh
Traceback (most recent call last):
File "config.py", line 49, in
external_validation_script='validate.sh')
TypeError: train() got an unexpected keyword argument 'saveto'
The text was updated successfully, but these errors were encountered: