08/12/2020

Designing Templates for Eliciting Commonsense Knowledge from Pretrained Sequence-to-Sequence Models

Jheng-Hong Yang, Sheng-Chieh Lin, Rodrigo Nogueira, Ming-Feng Tsai, Chuan-Ju Wang, Jimmy Lin

Keywords:

Abstract Paper Similar Papers

Abstract: While internalized “implicit knowledge” in pretrained transformers has led to fruitful progress in many natural language understanding tasks, how to most effectively elicit such knowledge remains an open question. Based on the text-to-text transfer transformer (T5) model, this work explores a template-based approach to extract implicit knowledge for commonsense reasoning on multiple-choice (MC) question answering tasks. Experiments on three representative MC datasets show the surprisingly good performance of our simple template, coupled with a logit normalization technique for disambiguation. Furthermore, we verify that our proposed template can be easily extended to other MC tasks with contexts such as supporting facts in open-book question answering settings. Starting from the MC task, this work initiates further research to find generic natural language templates that can effectively leverage stored knowledge in pretrained models.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6351-designing-templates-for-eliciting-commonsense-knowledge-from-pretrained-sequence-to-sequence-models
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 6:54 
 0
 0
 0
 1
 10:52 
 0
 0
 0
 0
 14:44 
 0
 0
 0
 0
 7:23 
 0
 0
 0
 0
 14:05 
 0
 0
 0
 0
 12:17 
 0
 0
 0
 0
 10:32 
 0
 0
 0
 0
 12:45 
 0
 0
 0
 0
 12:51 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 14:45 
 0
 0
 0
 0
 14:20 
 0
 0
 0
 0
 3:28 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 13:16 
 0
 0
 0
 0
 15:41 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 11:48 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 11:00 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 15:36 
 0
 0
 0
 0
 10:16 
 0
 0
 0
 0
 12:25 
 0
 0
 0
 0
 8:44 
 0
 0
 0
 0
 12:02 
 0
 0
 0
 0
 16:31 
 0
 0
 0
 0
 7:31 
 0
 0
 0
 0
 13:09 
 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 7:03 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 8:55 
 0
 0
 0
 0
 10:13 
 0
 0
 0
 0
 8:30 
 0
 0
 0
 0
 10:37 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 4:42 
 0
 0
 0
 0
 6:58