02/02/2021

Scene Graph Embeddings Using Relative Similarity Supervision

Paridhi Maheshwari, Ritwick Chaudhry, Vishwa Vinay

Keywords:

Abstract Paper Similar Papers

Abstract: Scene graphs are a powerful structured representation of the underlying content of images, and embeddings derived from them have been shown to be useful in multiple downstream tasks. In this work, we employ a graph convolutional network to exploit structure in scene graphs and produce image embeddings useful for semantic image retrieval. Different from classification-centric supervision traditionally available for learning image representations, we address the task of learning from relative similarity labels in a ranking context. Rooted within the contrastive learning paradigm, we propose a novel loss function that operates on pairs of similar and dissimilar images and imposes relative ordering between them in embedding space. We demonstrate that this Ranking loss, coupled with an intuitive triple sampling strategy, leads to robust representations that outperform well-known contrastive losses on the retrieval task. In addition, we provide qualitative evidence of how retrieved results that utilize structured scene information capture the global context of the scene, different from visual similarity search.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948276
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:09 
 0
 0
 0
 0
 5:28 
 0
 0
 0
 0
 16:52 
 0
 0
 0
 0
 7:43 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 13:59 
 0
 0
 0
 0
 0:58 
 0
 0
 0
 0
 9:46 
 0
 0
 0
 0
 15:08 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 14:28 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 6:58 
 0
 0
 0
 0
 3:47 
 0
 0
 0
 0
 15:25 
 0
 0
 0
 0
 9:20 
 0
 0
 0
 0
 3:01 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 2:01 
 0
 0
 0
 0
 2:57 
 0
 0
 0
 0
 3:10 
 0
 0
 0
 0
 11:35 
 0
 0
 0
 0
 3:04 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 11:20 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 17:04 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:08 
 0
 0
 0
 0
 9:14 
 0
 0
 0
 0
 15:40 
 0
 0
 0
 0
 4:47 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 15:24 
 0
 0
 0
 0
 11:37 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 11:33