08/12/2020

Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering

Wei Han, Hantao Huang, Tao Han

Keywords:

Abstract Paper Similar Papers

Abstract: Image text carries essential information to understand the scene and perform reasoning. Text-based visual question answering (text VQA) task focuses on visual questions that require reading text in images. Existing text VQA systems generate an answer by selecting from optical character recognition (OCR) texts or a fixed vocabulary. Positional information of text is underused and there is a lack of evidence for the generated answer. As such, this paper proposes a localization-aware answer prediction network (LaAP-Net) to address this challenge. Our LaAP-Net not only generates the answer to the question but also predicts a bounding box as evidence of the generated answer. Moreover, a context-enriched OCR representation (COR) for multimodal fusion is proposed to facilitate the localization task. Our proposed LaAP-Net outperforms existing approaches on three benchmark datasets for the text VQA task by a noticeable margin.

The video of this talk cannot be embedded. You can watch it here:
https://underline.io/lecture/6212-finding-the-evidence-localization-aware-answer-prediction-for-text-visual-question-answering
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at COLING 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 7:10 
 0
 0
 0
 0
 9:49 
 0
 0
 0
 0
 17:31 
 0
 0
 0
 0
 6:07 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 15:58 
 0
 0
 0
 0
 5:59 
 0
 0
 0
 0
 15:03 
 0
 0
 0
 0
 2:58 
 0
 0
 0
 0
 11:23 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 0:55 
 0
 0
 0
 0
 13:22 
 0
 0
 0
 0
 8:20 
 0
 0
 0
 0
 15:13 
 0
 0
 0
 0
 15:04 
 0
 0
 0
 0
 0:58 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 1:03 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 10:51 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 11:26 
 0
 0
 0
 0
 13:46 
 0
 0
 0
 0
 16:05 
 0
 0
 0
 0
 8:13 
 0
 0
 0
 0
 6:18 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 17:50 
 0
 0
 0
 0
 13:57 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 11:46 
 0
 0
 0
 0
 5:28 
 0
 0
 0
 0
 10:19 
 0
 0
 0
 0
 15:15 
 0
 0
 0
 0
 13:14