02/02/2021

VMLoc: Variational Fusion For Learning-Based Multimodal Camera Localization

Kaichen Zhou, Changhao Chen, Bing Wang, Muhamad Risqi U. Saputra, Niki Trigoni, Andrew Markham

Keywords:

Abstract Paper Similar Papers

Abstract: Recent learning-based approaches have achieved impressive results in the field of single-shot camera localization. However, how best to fuse multiple modalities (e.g., image and depth) and to deal with degraded or missing input are less well studied. In particular, we note that previous approaches towards deep fusion do not perform significantly better than models employing a single modality. We conjecture that this is because of the naive approaches to feature space fusion through summation or concatenation which do not take into account the different strengths of each modality. To address this, we propose an end-to-end framework, termed VMLoc, to fuse different sensor inputs into a common latent space through a variational Product-of-Experts (PoE) followed by attention-based fusion. Unlike previous multimodal variational works directly adapting the objective function of vanilla variational auto-encoder, we show how camera localization can be accurately estimated through an unbiased objective function based on importance weighting. Our model is extensively evaluated on RGB-D datasets and the results prove the efficacy of our model. The source code is available at https://github.com/Zalex97/VMLoc.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948314
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 16:59 
 0
 0
 0
 0
 14:48 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 13:20 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 11:24 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 5:28 
 0
 0
 0
 0
 14:37 
 0
 0
 0
 0
 15:23 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 15:15 
 0
 0
 0
 0
 10:00 
 0
 0
 0
 0
 16:35 
 0
 0
 0
 2
 4:57 
 0
 0
 0
 0
 5:51 
 0
 0
 0
 0
 1:03 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 9:07 
 0
 0
 0
 0
 15:04 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 14:17 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 4:54 
 0
 0
 0
 0
 10:10 
 0
 0
 0
 0
 4:38 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 11:43 
 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 13:34 
 0
 0
 0
 0
 16:27 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 9:34 
 0
 0
 0
 0
 8:48 
 0
 0
 0
 0
 4:13 
 0
 0
 0
 0
 4:43 
 0
 0
 0
 0
 1:01