23/08/2020

Taming pretrained transformers for extreme multi-label text classification

Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, Inderjit S. Dhillon

Keywords: extreme multi-label text classification, transformer models

Abstract Paper Similar Papers

Abstract: We consider the extreme multi-label text classification (XMC) problem: given an input text, return the most relevant labels from a large label collection. For example, the input text could be a product description on Amazon.com and the labels could be product categories. XMC is an important yet challenging problem in the NLP community. Recently, deep pretrained transformer models have achieved state-of-the-art performance on many NLP tasks including sentence classification, albeit with small label sets. However, naively applying deep transformer models to the XMC problem leads to sub-optimal performance due to the large output space and the label sparsity issue. In this paper, we propose X-Transformer, the first scalable approach to fine-tuning deep transformer models for the XMC problem. The proposed method achieves new state-of-the-art results on four XMC benchmark datasets. In particular, on a Wiki dataset with around 0.5 million labels, the prec@1 of X-Transformer is 77.28

The video of this talk cannot be embedded. You can watch it here:
https://dl.acm.org/doi/10.1145/3394486.3403368#sec-supp
(Link will open in new window)
 0
 0
 0
 1
  Share    
This is an embedded video. Talk and the respective paper are published at KDD 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 12:03 
 0
 0
 0
 0
 14:25 
 0
 0
 0
 0
 9:11 
 0
 0
 0
 0
 12:15 
 0
 0
 0
 0
 5:12 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 16:25 
 0
 0
 0
 0
 12:42 
 0
 0
 0
 0
 20:33 
 0
 0
 0
 0
 11:05 
 0
 0
 0
 0
 13:47 
 0
 0
 0
 0
 12:17 
 0
 0
 0
 0
 10:32 
 0
 0
 0
 0
 10:53 
 0
 0
 0
 0
 3:01 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 10:40 
 0
 0
 0
 0
 11:44 
 0
 0
 0
 0
 11:58 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 15:58 
 0
 0
 0
 0
 15:05 
 0
 0
 0
 0
 3:51 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 15:08 
 0
 0
 0
 0
 10:02 
 0
 0
 0
 0
 9:31 
 0
 0
 0
 0
 14:40 
 0
 0
 0
 0
 7:54 
 0
 0
 0
 0
 15:18 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 11:43 
 0
 0
 0
 0
 13:25 
 0
 0
 0
 0
 14:38 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 12:51 
 0
 0
 0
 0
 2:59 
 0
 0
 0
 0
 16:40 
 0
 0
 0
 0
 14:49