08/12/2020

Towards building a Robust Industry-scale Question Answering System

Rishav Chakravarti, Anthony Ferritto, Bhavani Iyer, Lin Pan, Radu Florian, Salim Roukos, Avi Sil

Keywords:

Abstract Paper Similar Papers

Abstract: Industry-scale NLP systems necessitate two features. 1. Robustness: “zero-shot transfer learning” (ZSTL) performance has to be commendable and 2. Efficiency: systems have to train efficiently and respond instantaneously. In this paper, we introduce the development of a production model called GAAMA (Go Ahead Ask Me Anything) which possess the above two characteristics. For robustness, it trains on the recently introduced Natural Questions (NQ) dataset. NQ poses additional challenges over older datasets like SQuAD: (a) QA systems need to read and comprehend an entire Wikipedia article rather than a small passage, and (b) NQ does not suffer from observation bias during construction, resulting in less lexical overlap between the question and the article. GAAMA consists of Attention-over-Attention, diversity among attention heads, hierarchical transfer learning, and synthetic data augmentation while being computationally inexpensive. Building on top of the powerful BERTQA model, GAAMA provides a ∼2.0% absolute boost in F1 over the industry-scale state-of-the-art (SOTA) system on NQ. Further, we show that GAAMA transfers zero-shot to unseen real life and important domains as it yields respectable performance on two benchmarks: the BioASQ and the newly introduced CovidQA datasets.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6118-towards-building-a-robust-industry-scale-question-answering-system
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING Workshops 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 14:48 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 9:09 
 0
 0
 0
 0
 13:51 
 0
 0
 0
 0
 9:40 
 0
 0
 0
 0
 22:38 
 0
 0
 0
 0
 21:03 
 0
 0
 0
 0
 6:59 
 0
 0
 0
 0
 4:24 
 0
 0
 0
 0
 4:33 
 0
 0
 0
 0
 15:25 
 0
 0
 0
 0
 14:20 
 0
 0
 0
 0
 16:37 
 0
 0
 0
 0
 8:30 
 0
 0
 0
 0
 14:52 
 0
 0
 0
 0
 20:33 
 0
 0
 0
 0
 6:15 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 19:50 
 0
 0
 0
 0
 16:43 
 0
 0
 0
 0
 10:28 
 0
 0
 0
 0
 14:42 
 0
 0
 0
 0
 13:30 
 0
 0
 0
 0
 13:06 
 0
 0
 0
 0
 3:28 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 0:57 
 0
 0
 0
 0
 11:49 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 4:49 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 16:52 
 0
 0
 0
 0
 8:45 
 0
 0
 0
 0
 6:58 
 0
 0
 0
 0
 11:23 
 0
 0
 0
 0
 14:48 
 0
 0
 0
 0
 14:41