08/12/2020

Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks

Lichao Sun, Congying Xia, Wenpeng Yin, Tingting Liang, Philip Yu, Lifang He

Keywords:

Abstract Paper Similar Papers

Abstract: Mixup is a latest data augmentation technique that linearly interpolates input examples and the corresponding labels. It has shown strong effectiveness in image classification by interpolating images at the pixel level. Inspired by this line of research, in this paper, we explore i) how to apply mixup to natural language processing tasks since text data can hardly be mixed in the raw format; ii) if mixup is still effective in transformer-based learning models,e.g., BERT.To achieve the goal, we incorporate mixup to transformer-based pre-trained architecture, named“mixup-transformer”, for a wide range of NLP tasks while keeping the whole end-to-end training system. We evaluate the proposed framework by running extensive experiments on the GLUEbenchmark. Furthermore, we also examine the performance of mixup-transformer in low-resource scenarios by reducing the training data with a certain ratio. Our studies show that mixup is a domain-independent data augmentation technique to pre-trained language models, resulting in significant performance improvement for transformer-based models.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6275-mixup-transformer-dynamic-data-augmentation-for-nlp-tasks
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 5:04 
 0
 0
 0
 0
 7:54 
 0
 0
 0
 0
 14:31 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 6:05 
 0
 0
 0
 0
 5:58 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 5:15 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 7:18 
 0
 0
 0
 0
 0:56 
 0
 0
 0
 0
 14:19 
 0
 0
 0
 0
 15:15 
 0
 0
 0
 0
 4:43 
 0
 0
 0
 0
 3:26 
 0
 0
 0
 0
 9:28 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 6:14 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 10:45 
 0
 0
 0
 0
 7:50 
 0
 0
 0
 0
 15:15 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 5:52 
 0
 0
 0
 0
 16:17 
 0
 0
 0
 0
 9:23 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 10:54 
 0
 0
 0
 0
 9:58 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 14:34 
 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 22:15 
 0
 0
 0
 0
 0:58 
 0
 0
 0
 0
 3:29 
 0
 0
 0
 0
 18:19 
 0
 0
 0
 0
 4:43