01/07/2020

Distill, Adapt, Distill: Training Small, In-Domain Models for Neural Machine Translation

Mitchell Gordon, Kevin Duh

Keywords:

Abstract Paper Similar Papers

Abstract: We explore best practices for training small, memory efficient machine translation models with sequence-level knowledge distillation in the domain adaptation setting. While both domain adaptation and knowledge distillation are widely-used, their interaction remains little understood. Our large-scale empirical results in machine translation (on three language pairs with three domains each) suggest distilling twice for best performance: once using general-domain data and again using in-domain data with an adapted teacher.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at ACL Workshops virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 12:12 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 5:58 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 6:03 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 14:20 
 0
 0
 0
 0
 10:47 
 0
 0
 0
 0
 7:40 
 0
 0
 0
 0
 12:02 
 0
 0
 0
 0
 14:44 
 0
 0
 0
 0
 11:35 
 0
 0
 0
 0
 14:21 
 0
 0
 0
 0
 11:41 
 0
 0
 0
 0
 19:42 
 0
 0
 0
 0
 15:19 
 0
 0
 0
 0
 15:17 
 0
 0
 0
 0
 6:56 
 0
 0
 0
 0
 7:26 
 0
 0
 0
 0
 11:42 
 0
 0
 0
 0
 11:44 
 0
 0
 0
 0
 13:28 
 0
 0
 0
 0
 15:09 
 0
 0
 0
 0
 4:42 
 0
 0
 0
 0
 11:06 
 0
 0
 0
 0
 10:02 
 0
 0
 0
 0
 6:28 
 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 4:15 
 0
 0
 0
 0
 12:03 
 0
 0
 0
 0
 12:20 
 0
 0
 0
 0
 11:43 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 2:22 
 0
 0
 0
 0
 14:31 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 5:09