05/12/2020

Can monolingual pretrained models help cross-lingual classification?

Zewen Chi, Li Dong, Furu Wei, Xianling Mao, Heyan Huang

Keywords:

Abstract Paper Similar Papers

Abstract: Multilingual pretrained language models (such as multilingual BERT) have achieved impressive results for cross-lingual transfer. However, due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. In this work, we present two approaches to improve zero-shot cross-lingual classification, by transferring the knowledge from monolingual pretrained models to multilingual ones. Experimental results on two cross-lingual classification benchmarks show that our methods outperform vanilla multilingual fine-tuning.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AACL 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 10:48 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 12:07 
 0
 0
 0
 0
 10:22 
 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 16:21 
 0
 0
 0
 0
 11:16 
 0
 0
 0
 0
 11:45 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 9:57 
 0
 0
 0
 0
 9:30 
 0
 0
 0
 0
 14:31 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 12:03 
 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 6:07 
 0
 0
 0
 0
 6:56 
 0
 0
 0
 0
 9:40 
 0
 0
 0
 0
 11:44 
 0
 0
 0
 0
 7:52 
 0
 0
 0
 0
 4:35 
 0
 0
 0
 0
 9:53 
 0
 0
 0
 0
 7:18 
 0
 0
 0
 0
 15:44 
 0
 0
 0
 0
 12:04 
 0
 0
 0
 0
 5:04 
 0
 0
 0
 0
 9:56 
 0
 0
 0
 0
 14:43 
 0
 0
 0
 0
 14:45 
 0
 0
 0
 0
 15:04 
 0
 0
 0
 0
 12:15 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 11:34 
 0
 0
 0
 0
 10:21 
 0
 0
 0
 0
 12:02 
 0
 0
 0
 0
 8:47 
 0
 0
 0
 0
 19:17 
 0
 0
 0
 0
 14:18 
 0
 0
 0
 0
 4:44