08/12/2020

SLICE: Supersense-based Lightweight Interpretable Contextual Embeddings

Cindy Aloui, Carlos Ramisch, Alexis Nasr, Lucie Barque

Keywords:

Abstract Paper Similar Papers

Abstract: Contextualised embeddings such as BERT have become de facto state-of-the-art references in many NLP applications, thanks to their impressive performances. However, their opaqueness makes it hard to interpret their behaviour. SLICE is a hybrid model that combines supersense labels with contextual embeddings. We introduce a weakly supervised method to learn interpretable embeddings from raw corpora and small lists of seed words. Our model is able to represent both a word and its context as embeddings into the same compact space, whose dimensions correspond to interpretable supersenses. We assess the model in a task of supersense tagging for French nouns. The little amount of supervision required makes it particularly well suited for low-resourced scenarios. Thanks to its interpretability, we perform linguistic analyses about the predicted supersenses in terms of input word and context representations.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6299-slice-supersense-based-lightweight-interpretable-contextual-embeddings
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 18:02 
 0
 0
 0
 0
 10:51 
 0
 0
 0
 0
 12:22 
 0
 0
 0
 0
 18:55 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 11:52 
 0
 0
 0
 0
 7:46 
 0
 0
 0
 0
 6:28 
 0
 0
 0
 0
 9:11 
 0
 0
 0
 0
 12:39 
 0
 0
 0
 0
 11:06 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 10:48 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 9:04 
 0
 0
 0
 0
 11:00 
 0
 0
 0
 0
 12:24 
 0
 0
 0
 0
 6:38 
 0
 0
 0
 0
 10:15 
 0
 0
 0
 0
 18:26 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 11:45 
 0
 0
 0
 0
 15:05 
 0
 0
 0
 0
 11:24 
 0
 0
 0
 0
 17:52 
 0
 0
 0
 0
 10:27 
 0
 0
 0
 0
 12:12 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 15:19 
 0
 0
 0
 0
 9:38 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 6:55 
 0
 0
 0
 0
 6:26 
 0
 0
 0
 0
 12:12 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 28:41 
 0
 0
 0
 0
 4:58