02/02/2021

Future-Guided Incremental Transformer for Simultaneous Translation

Shaolei Zhang, Yang Feng, Liangyou Li

Keywords:

Abstract Paper Similar Papers

Abstract: Simultaneous translation (ST) starts translations synchronously while reading source sentences, and is used in many online scenarios. The previous wait-k policy is concise and achieved good results in ST. However, wait-k policy faces two weaknesses: low training speed caused by the recalculation of hidden states and lack of future source information to guide training. For the low training speed, we propose an incremental Transformer with an average embedding layer (AEL) to accelerate the speed of calculation of the hidden states during training. For future-guided training, we propose a conventional Transformer as the teacher of the incremental Transformer, and try to invisibly embed some future information in the model through knowledge distillation. We conducted experiments on Chinese-English and German-English simultaneous translation tasks and compared with the wait-k policy to evaluate the proposed method. Our method can effectively increase the training speed by about 28 times on average at different k and implicitly embed some predictive abilities in the model, achieving better translation quality than wait-k baseline.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948467
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 19:42 
 0
 0
 0
 0
 4:51 
 0
 0
 0
 0
 15:07 
 0
 0
 0
 0
 9:23 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 10:41 
 0
 0
 0
 0
 19:32 
 0
 0
 0
 0
 9:36 
 0
 0
 0
 0
 14:06 
 0
 0
 0
 0
 7:05 
 0
 0
 0
 0
 8:59 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 6:40 
 0
 0
 0
 0
 6:03 
 0
 0
 0
 0
 7:17 
 0
 0
 0
 0
 15:57 
 0
 0
 0
 0
 4:15 
 0
 0
 0
 0
 3:52 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 12:32 
 0
 0
 0
 0
 14:52 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 0
 12:02 
 0
 0
 0
 0
 15:41 
 0
 0
 0
 0
 15:44 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 8:20 
 0
 0
 0
 0
 9:26 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 11:46 
 0
 0
 0
 0
 10:22 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 7:01 
 0
 0
 0
 0
 2:57 
 0
 0
 0
 0
 12:00 
 0
 0
 0
 0
 14:25 
 0
 0
 0
 0
 10:47 
 0
 0
 0
 0
 14:41 
 0
 0
 0
 0
 9:57 
 0
 0
 0
 0
 5:19