02/02/2021

Analysing the Noise Model Error for Realistic Noisy Label Data

Michael A. Hedderich, Dawei Zhu, Dietrich Klakow

Keywords:

Abstract Paper Similar Papers

Abstract: Distant and weak supervision allow to obtain large amounts of labeled training data quickly and cheaply, but these automatic annotations tend to contain a high amount of errors. A popular technique to overcome the negative effects of these noisy labels is noise modelling where the underlying noise process is modelled. In this work, we study the quality of these estimated noise models from the theoretical side by deriving the expected error of the noise model. Apart from evaluating the theoretical results on commonly used synthetic noise, we also publish NoisyNER, a new noisy label dataset from the NLP domain that was obtained through a realistic distant supervision technique. It provides seven sets of labels with differing noise patterns to evaluate different noise levels on the same instances. Parallel, clean labels are available making it possible to study scenarios where a small amount of gold-standard data can be leveraged. Our theoretical results and the corresponding experiments give insights into the factors that influence the noise model estimation like the noise distribution and the sampling technique.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948181
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 5:36 
 0
 0
 0
 0
 19:29 
 0
 0
 0
 0
 21:24 
 0
 0
 0
 0
 2:46 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 0
 10:56 
 0
 0
 0
 0
 12:03 
 0
 0
 0
 0
 5:20 
 0
 0
 0
 0
 14:19 
 0
 0
 0
 0
 20:38 
 0
 0
 0
 0
 9:42 
 0
 0
 0
 0
 3:15 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 13:00 
 0
 0
 0
 0
 11:43 
 0
 0
 0
 0
 14:49 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 10:37 
 0
 0
 0
 0
 9:53 
 0
 0
 0
 0
 5:13 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 8:12 
 0
 0
 0
 0
 3:03 
 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 5:16 
 0
 1
 0
 0
 4:46 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 14:48 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 4:29 
 0
 0
 0
 0
 4:51 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 14:30 
 0
 0
 0
 0
 12:02 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 5:24 
 0
 0
 0
 0
 5:06