Compatibility to hugging face and pre-training of these models

[filipelauarqantev]

Hi,

First congratulations for the implementation, I went throught the code and is very clean and well coded! I have some questions:

1- Once I train the model, can I use model.save_pretrained() and push to the hugging face hub?
2- Would be nice to try to add this implementation to hugging face Transformers library, have you thought about it?
3- All these models were previously pre-trained in a large corpus, you didn't do this step, right?

Again, thanks a lot for the implementation!