02/02/2021

Circles are like Ellipses, or Ellipses are like Circles? Measuring the Degree of Asymmetry of Static and Contextual Word Embeddings and the Implications to Representation Learning

Wei Zhang, Murray Campbell, Yang Yu, Sadhana Kumaravel

Keywords:

Abstract Paper Similar Papers

Abstract: Human judgments of word similarity have been a popular method of evaluating the quality of word embedding. But it fails to measure the geometry properties such as asymmetry. For example, it is more natural to say ``Ellipses are like Circles'' than ``Circles are like Ellipses''. Such asymmetry has been observed from the word evocation experiment, where one word is used to recall another. This association data have been understudied for measuring embedding quality. In this paper, we use three well-known evocation datasets for the purpose and study both static embedding as well as contextual embedding, such as BERT. To fight for the dynamic nature of BERT embedding, we probe BERT's conditional probabilities as a language model, using a large number of Wikipedia contexts to derive a theoretically justifiable Bayesian asymmetry score. The result shows that the asymmetry judgment and similarity judgments disagree, and asymmetry judgment aligns with its strong performance on ``extrinsic evaluations''. This is the first time we can show contextual embeddings's strength on intrinsic evaluation, and the asymmetry judgment provides a new perspective to evaluate contextual embedding and new insights for representation learning.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949338
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 6:28 
 0
 0
 0
 0
 18:55 
 0
 0
 0
 0
 9:11 
 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 12:03 
 0
 0
 0
 0
 11:36 
 0
 0
 0
 0
 12:47 
 0
 0
 0
 0
 11:45 
 0
 0
 0
 0
 6:14 
 0
 0
 0
 0
 18:56 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 12:07 
 0
 0
 0
 0
 7:02 
 0
 0
 0
 0
 12:19 
 0
 0
 0
 0
 9:57 
 0
 0
 0
 0
 4:31 
 0
 0
 0
 0
 17:06 
 0
 0
 0
 0
 12:22 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 15:00 
 0
 0
 1
 1
 15:14 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 12:44 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 3:10 
 0
 0
 0
 0
 7:46 
 0
 0
 0
 0
 10:12 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 15:05 
 0
 0
 0
 0
 17:52 
 0
 0
 0
 0
 11:20 
 0
 0
 0
 0
 9:04 
 0
 0
 0
 0
 12:18 
 0
 0
 0
 0
 9:54 
 0
 0
 0
 0
 9:45 
 0
 0
 0
 0
 12:39