02/02/2021

Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications

Haw-Shiuan Chang, Amol Agrawal, Andrew McCallum

Keywords:

Abstract Paper Similar Papers

Abstract: Most unsupervised NLP models represent each word with a single point or single region in semantic space, while the existing multi-sense word embeddings cannot represent longer word sequences like phrases or sentences. We propose a novel embedding method for a text sequence (a phrase or a sentence) where each sequence is represented by a distinct set of multi-mode codebook embeddings to capture different semantic facets of its meaning. The codebook embeddings can be viewed as the cluster centers which summarize the distribution of possibly co-occurring words in a pre-trained word embedding space. We introduce an end-to-end trainable neural model that directly predicts the set of cluster centers from the input text sequence during test time. Our experiments show that the per-sentence codebook embeddings significantly improve the performances in unsupervised sentence similarity and extractive summarization benchmarks. In phrase similarity experiments, we discover that the multi-facet embeddings provide an interpretable semantic representation but do not outperform the single-facet baseline.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949287
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 16:09 
 0
 0
 0
 0
 12:30 
 0
 0
 0
 0
 12:00 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 9:34 
 0
 0
 0
 0
 14:26 
 0
 0
 0
 0
 15:47 
 0
 0
 0
 0
 14:31 
 0
 0
 0
 0
 10:03 
 0
 0
 0
 0
 10:21 
 0
 0
 0
 0
 10:38 
 0
 0
 0
 0
 6:29 
 0
 0
 0
 0
 14:10 
 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 11:50 
 0
 0
 0
 0
 11:34 
 0
 0
 0
 0
 7:09 
 0
 0
 0
 0
 6:05 
 0
 0
 0
 0
 18:50 
 0
 0
 0
 0
 11:59 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 15:39 
 0
 0
 0
 0
 6:33 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 10:27 
 0
 0
 0
 0
 12:13 
 0
 0
 0
 0
 15:03 
 0
 0
 0
 0
 5:12 
 0
 0
 0
 0
 7:27 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 13:53 
 0
 0
 0
 0
 12:39 
 0
 0
 0
 0
 12:08 
 0
 0
 0
 0
 9:47 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 9:36 
 0
 0
 0
 0
 10:53 
 0
 0
 0
 0
 14:08 
 0
 0
 0
 0
 13:14