02/02/2021

UWSpeech: Speech to Speech Translation for Unwritten Languages

Chen Zhang, Xu Tan, Yi Ren, Tao Qin, Kejun Zhang, Tie-Yan Liu

Keywords:

Abstract Paper Similar Papers

Abstract: Existing speech to speech translation systems heavily rely on the text of target language: they usually translate source language either to target text and then synthesize target speech from text, or directly to target speech with target text for auxiliary training. However, those methods cannot be applied to unwritten target languages, which have no written text or phoneme available. In this paper, we develop a translation system for unwritten languages, named as UWSpeech, which converts target unwritten speech into discrete tokens with a converter, and then translates source-language speech into target discrete tokens with a translator, and finally synthesizes target speech from target discrete tokens with an inverter. We propose a method called XL-VAE, which enhances vector quantized variational autoencoder (VQ-VAE) with cross-lingual (XL) speech recognition, to train the converter and inverter of UWSpeech jointly. Experiments on Fisher Spanish-English conversation translation dataset show that UWSpeech outperforms direct translation and VQ-VAE baseline by about 16 and 10 BLEU points respectively, which demonstrate the advantages and potentials of UWSpeech.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949157
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 5:51 
 0
 0
 0
 0
 8:53 
 0
 0
 0
 0
 6:45 
 0
 0
 0
 0
 14:20 
 0
 0
 0
 0
 7:16 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 4:50 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 11:37 
 0
 0
 0
 0
 11:45 
 0
 0
 0
 0
 11:10 
 0
 0
 0
 0
 14:43 
 0
 0
 0
 0
 8:47 
 0
 0
 0
 0
 5:17 
 0
 0
 0
 0
 7:46 
 0
 0
 0
 0
 5:19 
 0
 0
 0
 0
 12:15 
 0
 0
 0
 0
 13:12 
 0
 0
 0
 0
 11:07 
 0
 0
 0
 0
 9:46 
 0
 0
 0
 0
 7:04 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 14:54 
 0
 0
 0
 0
 18:10 
 0
 0
 0
 0
 10:52 
 0
 0
 0
 0
 8:15 
 0
 0
 0
 0
 5:19 
 0
 0
 0
 0
 11:58 
 0
 0
 0
 0
 4:50 
 0
 0
 0
 0
 14:39 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 12:07 
 0
 0
 0
 0
 22:16 
 0
 0
 0
 0
 11:39 
 0
 0
 0
 0
 9:24 
 0
 0
 0
 0
 13:44 
 0
 0
 0
 0
 11:37 
 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 17:39