02/02/2021

Analogy Training Multilingual Encoders

Nicolas Garneau, Mareike Hartmann, Anders Sandholm, Sebastian Ruder, Ivan Vulić, Anders Søgaard

Keywords:

Abstract Paper Similar Papers

Abstract: Language encoders encode words and phrases in ways that capture their local semantic relatedness, but are known to be globally inconsistent. Global inconsistency can seemingly be corrected for, in part, by leveraging signals from knowledge bases, but previous results are partial and limited to monolingual English encoders. We extract a large-scale multilingual, multi-word analogy dataset from Wikidata for diagnosing and correcting for global inconsistencies, and then implement a four-way Siamese BERT architecture for grounding multilingual BERT (mBERT) in Wikidata through analogy training. We show that analogy training not only improves the global consistency of mBERT, as well as the isomorphism of language-specific subspaces, but also leads to consistent gains on downstream tasks such as bilingual dictionary induction and sentence retrieval.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949159
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 7:03 
 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 12:11 
 0
 0
 0
 0
 16:40 
 0
 0
 0
 0
 12:07 
 0
 0
 0
 0
 15:05 
 0
 0
 0
 0
 18:55 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 6:22 
 0
 0
 0
 0
 14:32 
 0
 0
 0
 0
 15:01 
 0
 0
 0
 0
 3:51 
 0
 0
 0
 0
 4:38 
 0
 0
 0
 0
 7:46 
 0
 0
 0
 0
 14:57 
 0
 0
 0
 0
 17:37 
 0
 0
 0
 0
 9:57 
 0
 0
 0
 0
 7:14 
 0
 0
 0
 0
 11:34 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 7:18 
 0
 0
 0
 0
 6:38 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 16:25 
 0
 0
 0
 0
 11:50 
 0
 0
 0
 0
 5:58 
 0
 0
 0
 0
 13:17 
 0
 0
 0
 0
 9:45 
 0
 0
 0
 0
 9:11 
 0
 0
 0
 0
 5:22 
 0
 0
 0
 0
 9:06 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 12:16 
 0
 0
 0
 0
 4:00 
 0
 0
 0
 0
 8:58 
 0
 0
 0
 0
 15:16 
 0
 0
 0
 0
 12:12 
 0
 0
 0
 0
 10:02 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 9:56