25/07/2020

DC-BERT: Decoupling question and document for efficient contextual encoding

Ping Nie, Yuyu Zhang, Xiubo Geng, Arun Ramamurthy, Le Song, Daxin Jiang

Keywords: open-domain question answering, document retrieval

Abstract Paper Similar Papers

Abstract: Recent studies on open-domain question answering have achieved prominent performance improvement using pre-trained language models such as BERT. State-of-the-art approaches typically follow the "retrieve and read" pipeline and employ BERT-based reranker to filter retrieved documents before feeding them into the reader module. The BERT retriever takes as input the concatenation of question and each retrieved document. Despite the success of these approaches in terms of QA accuracy, due to the concatenation, they can barely handle high-throughput of incoming questions each with a large collection of retrieved documents. To address the efficiency problem, we propose DC-BERT, a decoupled contextual encoding framework that has dual BERT models: an online BERT which encodes the question only once, and an offline BERT which pre-encodes all the documents and caches their encodings. On SQuAD Open and Natural Questions Open datasets, DC-BERT achieves 10x speedup on document retrieval, while retaining most (about 98

The video of this talk cannot be embedded. You can watch it here:
https://dl.acm.org/doi/10.1145/3397271.3401271#sec-supp
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at SIGIR 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 10:16 
 0
 0
 0
 1
 4:39 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 12:15 
 0
 0
 0
 0
 14:42 
 0
 0
 0
 0
 11:15 
 0
 0
 0
 0
 4:55 
 0
 0
 0
 0
 6:54 
 0
 0
 0
 0
 11:12 
 0
 0
 0
 0
 15:01 
 0
 0
 0
 0
 7:16 
 0
 0
 0
 0
 11:05 
 0
 0
 0
 0
 2:48 
 0
 0
 1
 1
 4:58 
 0
 0
 0
 0
 10:51 
 0
 0
 0
 0
 7:02 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 12:18 
 0
 0
 0
 0
 12:55 
 0
 0
 0
 0
 11:25 
 0
 0
 0
 0
 14:36 
 0
 0
 0
 0
 13:01 
 0
 0
 0
 0
 10:09 
 0
 0
 0
 0
 6:38 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 6:49 
 0
 0
 0
 0
 12:12 
 0
 0
 0
 0
 13:16 
 0
 0
 0
 0
 10:40 
 0
 0
 0
 0
 11:53 
 0
 0
 0
 0
 11:36 
 0
 0
 0
 0
 9:55 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 6:14 
 0
 0
 0
 0
 4:55 
 0
 0
 0
 0
 12:33 
 0
 0
 0
 0
 6:56 
 0
 0
 0
 0
 9:59 
 0
 0
 0
 0
 11:58