02/02/2021

Rethinking Bi-Level Optimization in Neural Architecture Search: A Gibbs Sampling Perspective

Chao Xue, Xiaoxing Wang, Junchi Yan, Yonggang Hu, Xiaokang Yang, Kewei Sun

Keywords:

Abstract Paper Similar Papers

Abstract: One-Shot architecture search, which aims to explore all possible operations jointly based on a single model, has been an active direction of Neural Architecture Search (NAS). As a well-known one-shot solution, Differentiable Architecture Search (DARTS) performs continuous relaxation on the architecture's importance and results in a bi-level optimization problem. However, as many recent studies have shown, DARTS cannot always work robustly for new tasks, which is mainly due to the approximate solution of the bi-level optimization. In this paper, one-shot neural architecture search is addressed by adopting a directed probabilistic graphical model to represent the joint probability distribution over data and model. Then, neural architectures are searched for and optimized by Gibbs sampling. We rethink the bi-level optimization problem as the task of Gibbs sampling from the posterior distribution, which expresses the preferences for different models given the observed dataset. We evaluate our proposed NAS method -- GibbsNAS on the search space used in DARTS/ENAS and the search space of NAS-Bench-201. Experimental results on multiple search space show the efficacy and stability of our approach.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948261
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 5:19 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 9:29 
 0
 0
 0
 0
 3:08 
 0
 0
 0
 0
 12:16 
 0
 0
 0
 1
 12:45 
 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 17:22 
 0
 0
 0
 0
 2:45 
 1
 0
 0
 0
 5:20 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 11:28 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 3:46 
 0
 0
 0
 0
 8:56 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 9:00 
 0
 0
 0
 0
 13:42 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 1:01 
 1
 0
 0
 0
 14:17 
 0
 0
 0
 0
 11:01 
 1
 0
 0
 1
 3:10 
 0
 0
 0
 0
 9:47 
 0
 0
 0
 0
 12:39 
 0
 0
 0
 0
 5:06 
 0
 0
 0
 0
 14:45 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 4:55 
 0
 0
 0
 0
 13:14 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 4:42 
 0
 0
 0
 0
 13:50 
 0
 0
 0
 0
 4:52 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 13:20