VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality

Abstract: Sketching in virtual reality (VR) enhances perception and understanding of 3D volumes, but is currently a challenging task, as spatial input devices (e.g., tracked controllers) do not provide any scaffolding or constraints for mid-air interaction. We present VRSketchIn, a VR sketching application using a 6DoF-tracked pen and a 6DoF-tracked tablet as input devices, combining unconstrained 3D mid-air with constrained 2D surface-based sketching. To explore what possibilities arise from this combination of 2D (pen on tablet) and 3D input (6DoF pen), we present a set of design dimensions and define the design space for 2D and 3D sketching interaction metaphors in VR. We categorize prior art inside our design space and implemented a subset of metaphors for pen and tablet sketching in our prototype. To gain a deeper understanding which specific sketching operations users perform with 2D and which with 3D metaphors, we present findings of usability walkthroughs with six participants.

06/12/2021

pose estimation, pose, neural rendering, zero-shot, shape learning, 3d reconstruction, datasets, generative models, multi-view, robotics

1:01

14/06/2020

generative models, inverse graphics, single-image 3d reconstruction, variational autoencoders, meshes, unsupervised learning

4:56

02/02/2021

Supervised Training of Dense Object Nets using Optimal Descriptors for Industrial Robotic Applications

Andras Gabor Kupcsik, Markus Spies, Alexander Klein and
Marco Todescato, Nicolai Waniek, Philipp Schillinger, Mathias Bürger

instance completion, shape completion, instance segmentation, 3d scene understanding, rgb-d, multi-view geometry, segmentation

1:01

26/04/2020

Fanbo Xiang, Yuzhe Qin, Kaichun Mo and
Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, Hao Su

Keywords Paper

simulation, manipulation, dataset, robotics, part, motion, segmentation, 3d

5:00

14/06/2020

3d dynamic scene reconstruction, shape completion, simultaneous localization and mapping in dynamic environments, physical scene reconstruction

1:01

25/04/2020

Mel Vecerik, Jean-Baptiste Regli, Oleg Sushkov and
David Barker, Rugile Pevceviciute, Thomas Roth ̈orl, Raia Hadsell, Lourdes Agapito, Jonathan Scholz

body modeling, 3d reconstruction, implicit function learning, differentiable rendering, 3d pose, monocular rgb, deep learning

1:01

05/01/2021

3d, human, clothing, texture, try-on, smpl, body, avatar, people, learning

1:01

02/02/2021