30/11/2020

DeepVoxels++: Enhancing the Fidelity of Novel View Synthesis from 3D Voxel Embeddings

Tong He, John Collomosse, Hailin Jin, Stefano Soatto

Keywords:

Abstract Paper Similar Papers

Abstract: We present a novel view synthesis method based upon latent voxel embeddings of an object, which encode both shape and appearance information and are learned without explicit 3D occupancy supervision. Our method uses an encoder-decoder architecture to learn such deep volumetric representations from a set of images taken at multiple viewpoints. Compared with DeepVoxels, our DeepVoxels++ applies a series of enhancements: a) a patch-based image feature extraction and neural rendering scheme that learns local shape and texture patterns, and enables neural rendering at high resolution; b) learned view-dependent feature transformation kernels to explicitly model perspective transformations induced by viewpoint changes; c) a recurrent-concurrent aggregation technique to alleviate single-view update bias of the voxel embeddings recurrent learning process. Combined with d) a simple yet effective implementation trick of frustum representation sufficient sampling, we achieve improved visual quality over the prior deep voxel-based methods (33% SSIM error reduction and 22% PSNR improvement) on 360-degree novel-view synthesis benchmarks.

The video of this talk cannot be embedded. You can watch it here:
https://accv2020.github.io/miniconf/poster_76.html
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at ACCV 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 4:41 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 4:48 
 0
 0
 0
 0
 4:34 
 0
 0
 0
 0
 1:01 
 0
 0
 1
 1
 5:08 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 6:47 
 0
 0
 0
 0
 17:23 
 0
 0
 0
 0
 10:00 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 14:21 
 0
 0
 0
 0
 7:33 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 3:23 
 0
 1
 0
 0
 5:06 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 15:15 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 2:49 
 0
 0
 0
 0
 3:11 
 0
 0
 0
 0
 17:44 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 14:54 
 0
 0
 0
 0
 14:42 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 4:30 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 4:34 
 0
 0
 0
 0
 14:11 
 0
 0
 0
 0
 10:51 
 0
 0
 0
 0
 4:54