02/02/2021

Object Relation Attention for Image Paragraph Captioning

Li-Chuan Yang, Chih-Yuan Yang, Jane Yung-jen Hsu

Keywords:

Abstract Paper Similar Papers

Abstract: Image paragraph captioning aims to automatically generate a paragraph from a given image. It is an extension of image captioning in terms of generating multiple sentences instead of a single one, and it is more challenging because paragraphs are longer, more informative, and more linguistically complicated. Because a paragraph consists of several sentences, an effective image paragraph captioning method should generate consistent sentences rather than contradictory ones. It is still an open question how to achieve this goal, and for it we propose a method to incorporate objects' spatial coherence into a language-generating model. For every two overlapping objects, the proposed method concatenates their raw visual features to create two directional pair features and learns weights optimizing those pair features as relation-aware object features for a language-generating model. Experimental results show that the proposed network extracts effective object features for image paragraph captioning and achieves promising performance against existing methods.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38947956
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 4:50 
 0
 0
 0
 0
 9:49 
 0
 0
 0
 0
 14:32 
 0
 0
 0
 0
 14:05 
 0
 0
 0
 0
 9:52 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 8:46 
 0
 0
 0
 0
 4:49 
 0
 0
 0
 0
 5:22 
 0
 0
 0
 0
 14:32 
 0
 0
 0
 0
 10:04 
 0
 0
 0
 0
 6:58 
 0
 0
 0
 0
 4:51 
 0
 0
 0
 0
 14:07 
 0
 0
 0
 0
 19:35 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 16:54 
 0
 0
 0
 0
 12:07 
 0
 0
 0
 0
 3:51 
 0
 0
 0
 0
 13:57 
 0
 0
 0
 0
 8:58 
 0
 0
 0
 0
 9:11 
 0
 0
 0
 0
 12:16 
 0
 0
 0
 0
 1:05 
 0
 0
 0
 0
 15:01 
 0
 0
 0
 0
 15:58 
 0
 0
 0
 0
 5:36 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 11:54 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 12:18 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 11:44 
 0
 0
 0
 0
 6:02 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 14:52 
 0
 0
 0
 0
 9:40 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 11:58