02/02/2021

Simple and Effective Stochastic Neural Networks

Tianyuan Yu, Yongxin Yang, Da Li, Timothy Hospedales, Tao Xiang

Keywords:

Abstract Paper Similar Papers

Abstract: Stochastic neural networks (SNNs) are currently topical, with several paradigms being actively investigated including dropout, Bayesian neural networks, variational information bottleneck (VIB) and noise regularized learning. These neural network variants impact several major considerations, including generalization, network compression, robustness against adversarial attack and label noise, and model calibration. However, many existing networks are complicated and expensive to train, and/or only address one or two of these practical considerations. In this paper we propose a simple and effective stochastic neural network (SE-SNN) architecture for discriminative learning by directly modeling activation uncertainty and encouraging high activation variability. Compared to existing SNNs, our SE-SNN is simpler to implement and faster to train, and produces state of the art results on network compression by pruning, adversarial defense, learning with label noise, and model calibration.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948469
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 15:01 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 16:26 
 0
 0
 0
 0
 4:46 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 13:40 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 3:20 
 0
 0
 1
 1
 6:15 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 8:30 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 4:34 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 3:58 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 2:55 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 8:13 
 0
 0
 0
 0
 8:38 
 1
 0
 0
 1
 12:23 
 0
 0
 0
 0
 10:07 
 0
 0
 0
 0
 18:14 
 0
 0
 0
 0
 4:52 
 0
 0
 0
 0
 12:31 
 0
 0
 0
 0
 6:26 
 0
 0
 0
 0
 14:59 
 0
 0
 0
 0
 14:24 
 0
 0
 0
 0
 14:30 
 0
 0
 0
 0
 5:10 
 0
 0
 0
 0
 4:25 
 0
 0
 0
 0
 8:20 
 0
 0
 0
 0
 4:12 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 4:10 
 0
 0
 0
 0
 15:06