Hierarchical Relational Inference

Abstract: Common-sense physical reasoning in the real world requires learning about the interactions of objects and their dynamics. The notion of an abstract object, however, encompasses a wide variety of physical objects that differ greatly in terms of the complex behaviors they support. To address this, we propose a novel approach to physical reasoning that models objects as hierarchies of parts that may locally behave separately, but also act more globally as a single whole. Unlike prior approaches, our method learns in an unsupervised fashion directly from raw visual images to discover objects, parts, and their relations. It explicitly distinguishes multiple levels of abstraction and improves over a strong baseline at modeling synthetic and real-world videos.

06/12/2021

action and behavior recognition, force prediction, human-object interaction, physical understanding, contact point prediction, motion and tracking, video understanding

4:59

14/06/2020

6d pose estimation, multi-object reasoning, object-level mapping, 3d reconstruction, multi-view reconstruction, volumetric reconstruction, slam, robotic manipulation

1:01

18/07/2021

Martin Sundermeyer, Maximilian Durner, En Yen Puang and
Zoltan-Csaba Marton, Narunas Vaskevicius, Kai O. Arras, Rudolph Triebel

Keywords Paper

object pose estimation, encodings, multi object, synthetic data, symmetries, autoencoder, embedding, 6d object detection, t-less, relative pose estimation

1:01

14/06/2020

hand-object reconstruction, pose estimation, object manipulation, photometric consistency, self-supervised learning, hands, objects, 3d reconstruction, manipulation

1:01

12/07/2020

Andy Zeng, Pete Florence, Jonathan Tompson and
Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, Johnny Lee

mot, multi-task learning, motion, affinity, attention, online

1:03

14/06/2020

Abhishek Das, Federico Carnevale, Hamza Merzic and
Laura Rimell, Rosalia Schneider, Josh Abramson, Alden Hung, Arun Ahuja, Stephen Clark, Greg Wayne, Feilx Hill

compositional action recognition, video understanding, something-something, action genome, charades, video transformer, multimodal fusion, spatial reasoning, spatio-temporal action recognition, revisiting spatio-temporal layouts

9:58

16/11/2020

Josh Merel, Saran Tunyasuvunakool, Arun Ahuja and
Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, Nicolas Heess

Action recognition, Fine-grained action, video attention, Spatial attention, Channel attention, Temporal attention, Spatio-temporal attention, Feature refinement

2:46

14/06/2020

parts model, lie group, rigid body, interpretable, bayesian, nonparametric, unsupervised, generative, dirichlet process, synthesis

4:58

05/01/2021

video captioning, spatio-temporal graph, video understanding, vision and language, knowledge distillation, transformer, computer vision.

1:01

14/06/2020

segmentation, video object segmentation, referring segmentation, referring video object segmentation, video object segmentation from referring expressions, referring image segmentation, referring image comprehension, optical flow, visual grounding

2:57