02/02/2021

A Hybrid Stochastic Gradient Hamiltonian Monte Carlo Method

Chao Zhang, Zhijian Li, Zebang Shen, Jiahao Xie, Hui Qian

Keywords:

Abstract Paper Similar Papers

Abstract: Recent theoretical analyses reveal that existing Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods need large mini-batches of samples (exponentially dependent on the dimension) to reduce the mean square error of gradient estimates and ensure non-asymptotic convergence guarantees when the target distribution has a nonconvex potential function. In this paper, we propose a novel SG-MCMC algorithm, called Hybrid Stochastic Gradient Hamiltonian Monte Carlo (HSG-HMC) method, which needs merely one sample per iteration and possesses a simple structure with only one hyperparameter. Such improvement leverages a hybrid stochastic gradient estimator that exploits historical stochastic gradient information to control the mean square error. Theoretical analyses show that our method obtains the best-known overall sample complexity to achieve epsilon-accuracy in terms of the 2-Wasserstein distance for sampling from distributions with nonconvex potential functions. Empirical studies on both simulated and real-world datasets demonstrate the advantage of our method.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948287
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 10:44 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 15:10 
 0
 0
 0
 0
 14:51 
 0
 0
 0
 0
 6:05 
 0
 0
 0
 0
 11:40 
 0
 0
 0
 0
 14:59 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 3:34 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 3:27 
 0
 0
 0
 0
 16:12 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 12:28 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 14:08 
 0
 0
 0
 0
 3:00 
 0
 0
 0
 0
 15:12 
 0
 0
 0
 0
 3:07 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 3:42 
 0
 0
 0
 0
 3:28 
 0
 0
 0
 0
 3:05 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 13:03 
 0
 0
 0
 0
 21:08 
 0
 0
 0
 0
 14:38 
 0
 0
 0
 0
 15:31 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 13:50 
 0
 0
 0
 0
 10:58 
 0
 0
 0
 0
 14:50 
 0
 0
 0
 0
 2:57 
 0
 0
 0
 0
 11:53 
 0
 0
 0
 0
 12:58 
 0
 0
 0
 0
 3:11 
 0
 0
 0
 0
 3:16