08/12/2020

Dynamic Curriculum Learning for Low-Resource Neural Machine Translation

Chen Xu, Bojie Hu, Yufan Jiang, Kai Feng, Zeyang Wang, Shen Huang, Qi Ju, Tong Xiao, Jingbo Zhu

Keywords:

Abstract Paper Similar Papers

Abstract: Large amounts of data has made neural machine translation (NMT) a big success in recent years. But it is still a challenge if we train these models on small-scale corpora. In this case, the way of using data appears to be more important. Here, we investigate the effective use of training data for low-resource NMT. In particular, we propose a dynamic curriculum learning (DCL) method to reorder training samples in training. Unlike previous work, we do not use a static scoring function for reordering. Instead, the order of training samples is dynamically determined in two ways - loss decline and model competence. This eases training by highlighting easy samples that the current model has enough competence to learn. We test our DCL method in a Transformer-based system. Experimental results show that DCL outperforms several strong baselines on three low-resource machine translation benchmarks and different sized data of WMT’16 En-De.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6343-dynamic-curriculum-learning-for-low-resource-neural-machine-translation
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 6:03 
 0
 0
 0
 0
 8:20 
 0
 0
 0
 0
 5:12 
 0
 0
 0
 0
 4:15 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 4:14 
 0
 0
 0
 0
 18:00 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 10:18 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 9:55 
 0
 0
 0
 0
 5:08 
 0
 0
 0
 0
 5:06 
 0
 0
 0
 0
 9:58 
 0
 0
 0
 0
 3:27 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 0:51 
 0
 0
 0
 0
 12:00 
 0
 0
 0
 0
 15:09 
 0
 0
 0
 0
 11:38 
 0
 0
 0
 0
 8:49 
 0
 0
 0
 0
 7:58 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 4:43 
 0
 0
 0
 0
 14:03 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 8:51 
 0
 0
 0
 0
 15:34 
 0
 0
 0
 0
 2:51 
 0
 0
 0
 0
 14:26 
 0
 0
 0
 0
 11:41 
 0
 0
 0
 0
 3:54 
 0
 0
 0
 0
 9:01 
 0
 0
 0
 0
 11:54 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 3:03 
 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 3:03 
 0
 0
 0
 0
 11:38