02/02/2021

Characterizing the Loss Landscape in Non-Negative Matrix Factorization

Johan Bjorck, Anmol Kabra, Kilian Q. Weinberger, Carla Gomes

Keywords:

Abstract Paper Similar Papers

Abstract: Non-negative matrix factorization (NMF) is a highly celebrated algorithm for matrix decomposition that guarantees non-negative factors. The underlying optimization problem is computationally intractable, yet in practice, gradient-descent-based methods often find good solutions. In this paper, we revisit the NMF optimization problem and analyze its loss landscape in non-worst-case settings. It has recently been observed that gradients in deep networks tend to point towards the final minimizer throughout the optimization procedure. We show that a similar property holds (with high probability) for NMF, provably in a non-worst case model with a planted solution, and empirically across an extensive suite of real-world NMF problems. Our analysis predicts that this property becomes more likely with growing number of parameters, and experiments suggest that a similar trend might also hold for deep neural networks---turning increasing dataset sizes and model sizes into a blessing from an optimization perspective.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38947901
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 5:25 
 0
 0
 0
 0
 3:33 
 0
 0
 0
 0
 3:05 
 0
 0
 0
 0
 5:16 
 0
 0
 0
 0
 13:48 
 1
 0
 0
 0
 14:01 
 0
 0
 0
 0
 12:32 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 9:51 
 0
 0
 0
 0
 9:24 
 0
 0
 0
 0
 5:20 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 4:53 
 0
 0
 0
 0
 14:59 
 0
 0
 0
 0
 3:42 
 0
 0
 0
 0
 14:46 
 0
 0
 0
 1
 12:45 
 0
 0
 0
 0
 12:16 
 0
 0
 0
 0
 19:34 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 5:05 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 13:13 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 11:32 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 5:15 
 0
 0
 0
 0
 4:34 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 7:57 
 0
 0
 0
 0
 14:36 
 0
 0
 0
 1
 3:28 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 15:33 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 8:56 
 0
 0
 0
 0
 4:37