08/12/2020

Learning Domain Terms - Empirical Methods to Enhance Enterprise Text Analytics Performance

Gargi Roy, Lipika Dey, Mohammad Shakir, Tirthankar Dasgupta

Keywords:

Abstract Paper Similar Papers

Abstract: Performance of standard text analytics algorithms are known to be substantially degraded on consumer generated data, which are often very noisy. These algorithms also do not work well on enterprise data which has a very different nature from News repositories, storybooks or Wikipedia data. Text cleaning is a mandatory step which aims at noise removal and correction to improve performance. However, enterprise data need special cleaning methods since it contains many domain terms which appear to be noise against a standard dictionary, but in reality are not so. In this work we present detailed analysis of characteristics of enterprise data and suggest unsupervised methods for cleaning these repositories after domain terms have been automatically segregated from true noise terms. Noise terms are thereafter corrected in a contextual fashion. The effectiveness of the method is established through careful manual evaluation of error corrections over several standard data sets, including those available for hate speech detection, where there is deliberate distortion to avoid detection. We also share results to show enhancement in classification accuracy after noise correction.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6120-learning-domain-terms---empirical-methods-to-enhance-enterprise-text-analytics-performance
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING Workshops 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 9:20 
 0
 0
 0
 0
 11:54 
 0
 0
 0
 0
 3:01 
 0
 0
 0
 0
 4:25 
 0
 0
 0
 0
 13:42 
 0
 0
 0
 0
 4:40 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:06 
 0
 0
 0
 0
 11:46 
 0
 0
 0
 0
 7:00 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 15:41 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 5:22 
 0
 0
 0
 0
 4:51 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 1
 4:49 
 0
 0
 0
 0
 2:31 
 0
 0
 0
 0
 14:00 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 10:37 
 0
 0
 0
 0
 11:45 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 14:10 
 0
 0
 0
 0
 15:39 
 0
 0
 0
 0
 10:10 
 0
 0
 0
 0
 11:39 
 0
 0
 0
 0
 12:04 
 0
 0
 0
 0
 14:35 
 0
 0
 0
 0
 15:25 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 9:19 
 0
 0
 0
 0
 7:17 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 10:59 
 0
 0
 0
 0
 13:50 
 0
 0
 0
 0
 15:11 
 0
 0
 0
 0
 28:41 
 0
 0
 0
 0
 15:03