02/02/2021

Humor Knowledge Enriched Transformer for Understanding Multimodal Humor

Md Kamrul Hasan, Sangwu Lee, Wasifur Rahman, Amir Zadeh, Rada Mihalcea, Louis-Philippe Morency, Ehsan Hoque

Keywords:

Abstract Paper Similar Papers

Abstract: Recognizing humor from a video utterance requires understanding the verbal and non-verbal components as well as incorporating the appropriate context and external knowledge. In this paper, we propose Humor Knowledge enriched Transformer (HKT) that can capture the gist of a multimodal humorous expression by integrating the preceding context and external knowledge. We incorporate humor centric external knowledge into the model by capturing the ambiguity and sentiment present in the language. We encode all the language, acoustic, vision, and humor centric features separately using Transformer based encoders, followed by a cross attention layer to exchange information among them. Our model achieves 77.36% and 79.41% accuracy in humorous punchline detection on UR-FUNNY and MUStaRD datasets -- achieving a new state-of-the-art on both datasets with the margin of 4.93% and 2.94% respectively. Furthermore, we demonstrate that our model can capture interpretable, humor-inducing patterns from all modalities.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949230
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 4:52 
 0
 0
 0
 0
 11:46 
 0
 0
 0
 0
 4:55 
 0
 0
 0
 0
 5:17 
 0
 0
 0
 0
 12:02 
 0
 0
 0
 0
 10:56 
 0
 0
 0
 0
 4:17 
 0
 0
 0
 0
 12:55 
 0
 0
 0
 0
 13:22 
 0
 0
 0
 0
 10:54 
 0
 0
 0
 0
 4:41 
 0
 0
 0
 0
 7:41 
 0
 0
 0
 0
 4:15 
 0
 0
 0
 0
 4:43 
 0
 0
 0
 0
 10:57 
 0
 0
 0
 0
 12:15 
 0
 0
 0
 0
 9:48 
 0
 0
 0
 0
 13:00 
 0
 0
 0
 0
 9:50 
 0
 0
 0
 0
 14:38 
 0
 0
 0
 0
 7:03 
 0
 0
 0
 0
 13:03 
 0
 0
 0
 0
 10:01 
 0
 0
 0
 0
 19:57 
 0
 0
 0
 0
 13:03 
 0
 0
 0
 0
 10:40 
 0
 0
 0
 0
 12:13 
 0
 0
 0
 0
 12:35 
 0
 0
 0
 0
 10:25 
 0
 0
 0
 0
 19:41 
 0
 0
 0
 0
 4:51 
 0
 0
 0
 0
 10:27 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 15:43 
 0
 0
 0
 0
 7:51 
 0
 0
 0
 0
 18:15 
 0
 0
 0
 0
 14:20 
 0
 0
 0
 0
 4:53 
 0
 0
 0
 0
 12:18 
 0
 0
 0
 0
 11:05