Learning 3D Face Reconstruction with a Pose Guidance Network

Abstract: We present a self-supervised learning approach to learning monocular 3D face reconstruction with a pose guidance network (PGN). First, we unveil the bottleneck of pose estimation in prior parametric 3D face learning methods, and propose to utilize 3D face landmarks for estimating pose parameters. With our specially designed PGN, our model can learn from both faces with fully labeled 3D landmarks and unlimited unlabeled in-the-wild face images. Our network is further augmented with a self-supervised learning scheme, which exploits face geometry information embedded in multiple frames of the same person, to alleviate the ill-posed nature of regressing 3D face geometry from a single image. These three insights yield a single approach that combines the complementary strengths of parametric model learning and data-driven learning techniques. We conduct a rigorous evaluation on the challenging AFLW2000-3D, Florence and FaceWarehouse datasets, and show that our method outperforms the state-of-the-art for all metrics.

02/02/2021

3d human pose estimation, self-supervised learning, disentangling factors of variation, human puppet model, pose transfer, novel view synthesis, human part segmentation

5:00

14/06/2020

self-supervised, unsupervised, keypoints, landmarks, pose, videos, adversarial, gan, disentanglement, factorizations

5:01

14/06/2020

Deep 3D Portrait From a Single Image

Sicheng Xu, Jiaolong Yang, Dong Chen and
Fang Wen, Yu Deng, Yunde Jia, Xin Tong

deblurring, face, multi-view, video, blur, GAN, novel view synthesis, inversion, deep learning, dataset

2:50

14/06/2020

hand-object reconstruction, pose estimation, object manipulation, photometric consistency, self-supervised learning, hands, objects, 3d reconstruction, manipulation

1:01

14/06/2020

differentiable rendering, signed distance field, image-based 3d reconstruction, 3d shape optimization, deep learning, inverse graphics

5:01

14/06/2020

Jogendra Nath Kundu, Siddharth Seth, Anirudh Jamkhandi and
Pradyumna YM, Varun Jampani, Anirban Chakraborty, Venkatesh Babu R

structured representation, 3D representation, 3D Gaussians, image generation, image synthesis, image editing, controlled generation, GANs

2:49

18/07/2021

optical flow, stereo matching, self-supervised learning, unsupervised learning, geometric constraints, convolutional neural networks (cnns), motion, depth

1:01

06/12/2020