30/11/2020

Multi-Task Learning for Simultaneous Video Generation and Remote Photoplethysmography Estimation

Yun-Yun Tsou, Yi-An Lee, Chiou-Ting Hsu

Keywords:

Abstract Paper Similar Papers

Abstract: Remote photoplethysmography (rPPG) is a contactless method for estimating physiological signals from facial videos. Without large supervised datasets, learning a robust rPPG estimation model is extremely challenging. Instead of merely focusing on model learning, we believe data augmentation may be of greater importance for this task. In this paper, we propose a novel multi-task learning framework to simultaneously augment training data while learning the rPPG estimation model. We design three joint-learning networks: rPPG estimation network, Image-to-Video network, and Video-to-Video network, to estimate rPPG signals from face videos, to generate synthetic videos from a source image and a specified rPPG signal, and to generate synthetic videos from a source video and a specified rPPG signal, respectively. Experimental results on three benchmark datasets, COHFACE, UBFC, and PURE, show that our method successfully generates photo-realistic videos and significantly outperforms existing methods with a large margin.

The video of this talk cannot be embedded. You can watch it here:
https://accv2020.github.io/miniconf/poster_330.html
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at ACCV 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 2:50 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:04 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:13 
 0
 0
 0
 0
 8:29 
 0
 0
 0
 0
 9:07 
 0
 0
 0
 0
 6:53 
 0
 0
 0
 0
 6:14 
 0
 0
 0
 0
 11:36 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 2:46 
 0
 0
 0
 0
 9:47 
 0
 0
 0
 0
 4:34 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 14:14 
 0
 0
 0
 0
 0:57 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 2:46 
 0
 0
 0
 0
 12:29 
 0
 0
 0
 0
 9:59 
 0
 0
 0
 0
 3:03 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 15:23 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 19:47 
 0
 0
 0
 0
 9:32 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 12:03 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 5:00