Knock on Wood: Combining Redirected Touching and Physical Props for Tool-Based Interaction in Virtual Reality

Abstract: When physical props serve as proxies for virtual tools used to manipulate the virtual environment, it is challenging to provide appropriate haptic feedback. Redirected tool-mediated manipulation addresses this challenge by distorting the mapping between physical and virtual tools to provide a sensation of manipulating the virtual environment, when the physical tool comes into contact with another physical prop. For example, a virtual hammer’s position can be offset to ensure that physical impacts accompany each strike of a virtual nail. We demonstrate the idea by showing that it can be used to create sensations of impact and resistance when driving a virtual nail into a surface, when tightening a virtual screw, and when sawing through a virtual plank. The results of a user study indicate that the proposed approach is perceived as more realistic than interaction with a single physical prop or controller and no notable detriments to precision were observed.

25/04/2020

Andras Gabor Kupcsik, Markus Spies, Alexander Klein and
Marco Todescato, Nicolai Waniek, Philipp Schillinger, Mathias Bürger

Yuntao Wang, Zichao Chen, Hanchuan Li and
Zhengyi Cao, Huiyi Luo, Tengxiang Zhang, Ke Ou, John Raiti, Chun Yu, Shwetak Patel, Yuanchun Shi

Wanli Peng, Hao Pan, He Liu, Yi Sun

hand-object reconstruction, pose estimation, object manipulation, photometric consistency, self-supervised learning, hands, objects, 3d reconstruction, manipulation

1:01

14/06/2020

Applications, Applications, Computer Vision; Deep Learning, Deep Autoencoders; Deep Learning, Generative Models; Probabilistic Methods , Reinforcement Learning and Planning, Deep RL

5:13

06/12/2020

6d pose estimation, multi-object reasoning, object-level mapping, 3d reconstruction, multi-view reconstruction, volumetric reconstruction, slam, robotic manipulation

1:01

16/11/2020

Transporter Networks: Rearranging the Visual World for Robotic Manipulation

Andy Zeng, Pete Florence, Jonathan Tompson and
Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, Johnny Lee

Roland Hafner, Tim Hertweck, Philipp Kloeppner and
Michael Bloesch, Michael Neunert, Markus Wulfmeier, Saran Tunyasuvunakool, Nicolas Heess, Martin Riedmiller

Keywords Paper

5:24

16/11/2020

S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistency

Mel Vecerik, Jean-Baptiste Regli, Oleg Sushkov and
David Barker, Rugile Pevceviciute, Thomas Roth ̈orl, Raia Hadsell, Lourdes Agapito, Jonathan Scholz

6d pose estimation, keypoints detection, relative pose, object detection, deep learning, computer vision, multi-task learning, metric learning, multi-view learning, epipolar geometry

1:01

17/08/2020

hand pose estimation, 3d pose estimation, hand-object interaction, weak supervision, domain adaptation, generative adversarial network, 3d mesh model, mano, hand gesture recognition, data synthesis

5:01

06/12/2020

Learning Physical Common Sense as Knowledge Graph Completion via BERT Data Augmentation and Constrained Tucker Factorization

button, haptic, modeling, simulation, tactility, force feedback, vibration, input device, haptic rendering, fd model, fdvv model

13:59

16/11/2020