08/12/2020

SWAFN: Sentimental Words Aware Fusion Network for Multimodal Sentiment Analysis

Minping Chen, Xia Li

Keywords:

Abstract Paper Similar Papers

Abstract: Multimodal sentiment analysis aims to predict sentiment of language text with the help of other modalities, such as vision and acoustic features. Previous studies focused on learning the joint representation of multiple modalities, ignoring some useful knowledge contained in language modal. In this paper, we try to incorporate sentimental words knowledge into the fusion network to guide the learning of joint representation of multimodal features. Our method consists of two components: shallow fusion part and aggregation part. For the shallow fusion part, we use crossmodal coattention mechanism to obtain bidirectional context information of each two modals to get the fused shallow representations. For the aggregation part, we design a multitask of sentimental words classification to help and guide the deep fusion of the three modalities and obtain the final sentimental words aware fusion representation. We carry out several experiments on CMU-MOSI, CMU-MOSEI and YouTube datasets. The experimental results show that introducing sentimental words prediction as a multitask can really improve the fusion representation of multiple modalities.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6151-swafn-sentimental-words-aware-fusion-network-for-multimodal-sentiment-analysis
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 10:03 
 0
 0
 0
 0
 11:51 
 0
 0
 0
 0
 14:32 
 0
 0
 0
 0
 17:44 
 0
 0
 0
 0
 14:34 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 3:38 
 0
 0
 0
 0
 13:27 
 0
 0
 0
 0
 10:46 
 0
 0
 0
 0
 10:46 
 0
 0
 0
 0
 9:57 
 0
 0
 0
 0
 13:34 
 0
 0
 0
 0
 10:19 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 7:33 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 11:46 
 0
 0
 0
 0
 14:50 
 0
 0
 0
 0
 15:17 
 0
 0
 0
 0
 14:35 
 0
 0
 0
 0
 14:33 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 12:44 
 0
 0
 0
 0
 2:57 
 0
 0
 0
 0
 4:49 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 6:47 
 0
 0
 0
 0
 14:05 
 0
 0
 0
 0
 12:38 
 0
 0
 0
 0
 11:45 
 0
 0
 0
 0
 6:46 
 0
 0
 0
 0
 18:42 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 1
 11:06 
 0
 0
 0
 0
 14:12 
 0
 0
 0
 0
 2:56