30/11/2020

Visually Guided Sound Source Separation using Cascaded Opponent Filter Network

Lingyu Zhu, Esa Rahtu

Keywords:

Abstract Paper Similar Papers

Abstract: The objective of this paper is to recover the original component signals from a mixture audio with the aid of visual cues of the sound sources. Such task is usually referred as visually guided sound source separation. The proposed Cascaded Opponent Filter (COF) framework consists of multiple stages, which recursively refine the source separation. A key element in COF is a novel opponent filter module that identifies and relocates residual components between sources. The system is guided by the appearance and motion of the source, and, for this purpose, we study different representations based on video frames, optical flows, dynamic images, and their combinations. Finally, we propose a Sound Source Location Masking (SSLM) technique, which, together with COF, produces a pixel level mask of the source location. The entire system is trained in an end-to-end manner using a large set of unlabelled videos. We compare COF with recent baselines and obtain the state-of-the-art performance in three challenging datasets (MUSIC, A-MUSIC, and A-NATURAL).

The video of this talk cannot be embedded. You can watch it here:
https://accv2020.github.io/miniconf/poster_293.html
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at ACCV 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 6:40 
 0
 0
 0
 0
 9:44 
 0
 0
 0
 0
 4:41 
 0
 0
 0
 0
 5:36 
 0
 0
 0
 0
 10:42 
 0
 0
 0
 0
 8:46 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 9:52 
 0
 0
 0
 0
 7:38 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 3:05 
 0
 0
 0
 0
 1:02 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 3:11 
 0
 0
 0
 0
 4:34 
 0
 0
 0
 0
 4:41 
 0
 0
 0
 0
 14:55 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 2:55 
 0
 0
 0
 0
 0:56 
 0
 0
 0
 0
 15:15 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 14:19 
 0
 0
 0
 0
 17:41 
 0
 0
 0
 0
 7:14 
 0
 0
 0
 0
 2:59 
 0
 0
 0
 0
 2:57 
 0
 0
 0
 0
 12:27 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 18:17 
 0
 0
 0
 0
 9:52 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 15:23