08/12/2020

Noise Isn’t Always Negative: Countering Exposure Bias in Sequence-to-Sequence Inflection Models

Garrett Nicolai, Miikka Silfverberg

Keywords:

Abstract Paper Similar Papers

Abstract: Morphological inflection, like many sequence-to-sequence tasks, sees great performance from recurrent neural architectures when data is plentiful, but performance falls off sharply in lower-data settings. We investigate one aspect of neural seq2seq models that we hypothesize contributes to overfitting - teacher forcing. By creating different training and test conditions, exposure bias increases the likelihood that a system too closely models its training data. Experiments show that teacher-forced models struggle to recover when they enter unknown territory. However, a simple modification to the training algorithm to more closely mimic test conditions creates models that are better able to generalize to unseen environments.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6227-noise-isn%27t-always-negative-countering-exposure-bias-in-sequence-to-sequence-inflection-models
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 8:29 
 0
 0
 0
 0
 21:39 
 0
 0
 0
 0
 20:50 
 0
 0
 0
 0
 6:14 
 0
 0
 0
 0
 14:37 
 0
 0
 0
 0
 14:34 
 0
 0
 0
 0
 5:05 
 0
 0
 0
 0
 15:48 
 0
 0
 0
 0
 14:21 
 0
 0
 0
 0
 2:56 
 0
 0
 0
 0
 10:57 
 0
 0
 0
 0
 4:42 
 0
 0
 0
 0
 10:39 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 15:14 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 14:02 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 5:19 
 0
 0
 0
 0
 14:12 
 0
 0
 0
 0
 10:19 
 0
 0
 0
 0
 3:27 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 3:26 
 0
 0
 0
 0
 15:03 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 0
 9:38 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 14:01 
 0
 0
 0
 0
 5:04 
 0
 0
 0
 0
 8:28 
 0
 0
 0
 0
 10:03 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 8:29 
 0
 0
 0
 0
 17:11 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 2:22 
 0
 0
 0
 0
 12:16 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 3:21