You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, I have implemented the mBART (span denoising) and mT5 (span prediction) pre-training approaches but according to the ULL2 paper (https://arxiv.org/pdf/2205.05131.pdf) a more comprehensive mixture of denoisers would help a lot.
Currently, you may use either mT5 or mBART style but I would like to enable the user to specify a comma separated list of denoising objectives and a comma separated list of the probabilities of using these objectives along with requisite hyperparams for each objective. If this is done we can play with some cool stuff.
The text was updated successfully, but these errors were encountered:
Currently, I have implemented the mBART (span denoising) and mT5 (span prediction) pre-training approaches but according to the ULL2 paper (https://arxiv.org/pdf/2205.05131.pdf) a more comprehensive mixture of denoisers would help a lot.
Currently, you may use either mT5 or mBART style but I would like to enable the user to specify a comma separated list of denoising objectives and a comma separated list of the probabilities of using these objectives along with requisite hyperparams for each objective. If this is done we can play with some cool stuff.
The text was updated successfully, but these errors were encountered: