2021-EMNLP-Improving Neural Machine Translation by Bidirectional Training

[thangk]

Link: arXiv

Main problem

Improving the performance of NMT model translation is hard. Even when there are high-performing models, they're often configured with much more complicated settings. That limits its applicability to a broader range of tasks. Thus, there needs to be a solution that's simple and yet can be applied to more tasks.

Proposed method

The author proposes a method that involves pre-training the data that goes from source->target then to source+target->target+source. This doubles the training data. The model is then tuned for the normal direction of source->target direction.

My Summary

According to the paper's result, this proposed method significantly improved the performance than a regular one-directional pre-training approach. The author observes the best BLEU scores in IWSL21 in low-resource tasks. The author claims this is a better and simple bilingual code-switching approach and also improves bilingual alignment quality. There are more testing needed to be done, such as if this bidirectional pre-training can be applied to previous NMT systems.

Datasets

IWSLT21 EN-DE
WMT16 EN-RO
WMT19 EN-GU
IWSLT21 EN-SW
WMT14 EN-DE
WMT19 EN-DE
WMT17 ZH-EN
WAT17 JA-EN