02/02/2021

DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances

Xiaodong Gu, Kang Min Yoo, Jung-Woo Ha

Keywords:

Abstract Paper Similar Papers

Abstract: Recent advances in pre-trained language models have significantly improved neural response generation. However, existing methods usually view the dialogue context as a linear sequence of tokens and learn to generate the next word through token-level self-attention. Such token-level encoding hinders the exploration of discourse-level coherence among utterances. This paper presents DialogBERT, a novel conversational response generation model that enhances previous PLM-based dialogue models. DialogBERT employs a hierarchical Transformer architecture. To efficiently capture the discourse-level coherence among utterances, we propose two training objectives, including masked utterance regression and distributed utterance order ranking in analogy to the original BERT training. Experiments on three multi-turn conversation datasets show that our approach remarkably outperforms three baselines, such as BART and DialoGPT, in terms of quantitative evaluation. The human evaluation suggests that DialogBERT generates more coherent, informative, and human-like responses than the baselines with significant margins.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949285
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 11:26 
 0
 0
 0
 0
 8:08 
 0
 0
 0
 0
 4:43 
 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 11:41 
 0
 0
 0
 0
 11:18 
 0
 0
 0
 0
 11:09 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 13:14 
 0
 0
 0
 0
 19:17 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 10:27 
 0
 0
 0
 0
 10:53 
 0
 0
 0
 0
 10:15 
 0
 0
 0
 0
 3:51 
 0
 0
 0
 0
 12:06 
 0
 0
 0
 0
 7:02 
 0
 0
 0
 0
 7:10 
 0
 0
 0
 0
 6:06 
 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 13:50 
 0
 0
 0
 0
 17:37 
 0
 0
 0
 0
 12:00 
 0
 0
 0
 0
 10:16 
 0
 0
 0
 0
 7:09 
 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 11:51 
 0
 0
 0
 0
 11:43 
 0
 0
 0
 0
 12:45 
 0
 0
 0
 0
 10:53 
 0
 0
 0
 1
 10:52 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 7:16 
 0
 0
 0
 0
 13:44 
 0
 0
 0
 0
 11:19 
 0
 0
 0
 0
 11:37 
 0
 0
 0
 0
 11:50 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 11:37 
 0
 0
 0
 0
 6:48