08/12/2020

Incremental Neural Lexical Coherence Modeling

Sungho Jeon, Michael Strube

Keywords:

Abstract Paper Similar Papers

Abstract: Pretrained language models, neural models pretrained on massive amounts of data, have established the state of the art in a range of NLP tasks. They are based on a modern machine-learning technique, the Transformer which relates all items simultaneously to capture semantic relations in sequences. However, it differs from what humans do. Humans read sentences one-by-one, incrementally. Can neural models benefit by interpreting texts incrementally as humans do? We investigate this question in coherence modeling. We propose a coherence model which interprets sentences incrementally to capture lexical relations between them. We compare the state of the art in each task, simple neural models relying on a pretrained language model, and our model in two downstream tasks. Our findings suggest that interpreting texts incrementally as humans could be useful to design more advanced models.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6243-incremental-neural-lexical-coherence-modeling
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 12:06 
 0
 0
 0
 0
 9:42 
 0
 0
 0
 0
 11:05 
 0
 0
 0
 0
 6:56 
 0
 0
 0
 0
 12:00 
 0
 0
 0
 0
 12:06 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 19:17 
 0
 0
 0
 0
 16:04 
 0
 0
 0
 0
 9:58 
 0
 0
 0
 0
 6:45 
 0
 0
 0
 0
 14:05 
 0
 0
 0
 0
 11:04 
 0
 0
 0
 0
 12:17 
 0
 0
 0
 0
 13:53 
 0
 0
 0
 0
 9:21 
 0
 0
 0
 0
 13:17 
 0
 0
 0
 0
 10:57 
 0
 0
 0
 0
 18:42 
 0
 0
 0
 0
 4:50 
 0
 0
 0
 0
 7:02 
 0
 0
 0
 0
 10:10 
 0
 0
 0
 0
 10:53 
 0
 0
 0
 0
 11:37 
 0
 0
 0
 0
 7:17 
 0
 0
 0
 0
 13:55 
 0
 0
 0
 0
 11:29 
 0
 0
 0
 0
 5:15 
 0
 0
 0
 0
 6:48 
 0
 0
 0
 0
 11:48 
 0
 0
 0
 0
 14:50 
 0
 0
 0
 0
 10:29 
 0
 0
 0
 0
 11:31 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 7:22 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 1
 10:41 
 0
 0
 0
 0
 13:52 
 0
 0
 0
 0
 6:33