06/12/2021

COHESIV: Contrastive Object and Hand Embedding Segmentation In Video

Dandan Shan, Richard Higgins, David Fouhey

Keywords: deep learning, contrastive learning

Abstract Paper Similar Papers

Abstract: In this paper we learn to segment hands and hand-held objects from motion. Our system takes a single RGB image and hand location as input to segment the hand and hand-held object. For learning, we generate responsibility maps that show how well a hand's motion explains other pixels' motion in video. We use these responsibility maps as pseudo-labels to train a weakly-supervised neural network using an attention-based similarity loss and contrastive loss. Our system outperforms alternate methods, achieving good performance on the 100DOH, EPIC-KITCHENS, and HO3D datasets.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at NeurIPS 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 14:38 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 5:17 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 3:34 
 0
 0
 0
 0
 6:13 
 0
 0
 0
 0
 5:30 
 0
 0
 0
 0
 12:36 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 6:29 
 0
 0
 0
 0
 5:13 
 0
 0
 0
 0
 2:55 
 0
 0
 0
 0
 9:15 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:36 
 0
 0
 0
 0
 3:03 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 9:50 
 0
 0
 0
 0
 4:51 
 0
 0
 0
 1
 5:13 
 0
 0
 0
 0
 12:28 
 0
 0
 0
 0
 6:46 
 0
 0
 0
 0
 17:04 
 0
 0
 0
 0
 12:47 
 0
 0
 0
 0
 10:46 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 9:32 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 8:29 
 0
 0
 0
 0
 8:34 
 0
 1
 1
 1
 1:01 
 0
 0
 0
 0
 4:56