Fast Training of Provably Robust Neural Networks by SingleProp

02/02/2021

Fast Training of Provably Robust Neural Networks by SingleProp

Akhilan Boopathy, Lily Weng, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, Luca Daniel

Keywords:

Abstract Paper Similar Papers

Abstract: Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees. However, these techniques can be computationally costly due to the use of certification during training. We develop a new regularizer that is both more efficient than existing certified defenses, requiring only one additional forward propagation through a network, and can be used to train networks with similar certified accuracy. Through experiments on MNIST and CIFAR-10 we demonstrate improvements in training speed and comparable certified accuracy compared to state-of-the-art certified defenses.

The video of this talk cannot be embedded. You can watch it here:

https://slideslive.com/38949255

(Link will open in new window)

0

0

0

0

Share

This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment

no comments yet

Similar Papers

14/06/2020

Computing the Testing Error Without a Testing Set

Ciprian A. Corneanu, Sergio Escalera, Aleix M. Martinez

Keywords Paper

deep learning, algebraic topology, generalization, object recognition, facial analysis, semantic segmentation

0

0

0

0

4:43

19/08/2021

Safety Analysis of Deep Neural Networks

Dario Guidotti

Keywords Paper

AI Ethics, Trust, Fairness, Trustable Learning, Validation and Verification, Deep Learning, Adversarial Machine Learning

0

0

0

0

11:33

06/12/2021

Towards Efficient and Effective Adversarial Training

Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, Venkatesh Babu R

Keywords Paper

deep learning, optimization, robustness, adversarial robustness and security

0

0

0

0

14:24

02/02/2021

Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks

Xin Chen, Lingxi Xie, Jun Wu and
Longhui Wei, Yuhui Xu, Qi Tian

Keywords Paper

0

0

0

0

15:02

05/04/2021

Pipelined Backpropagation at Scale: Training Large Models without Batches

Atli Kosson, Vitaliy Chiley, Abhi Venigalla and
Joel Hestness, Urs Koster

Keywords Paper

0

0

0

0

4:14

05/04/2021

Pipelined Backpropagation at Scale: Training Large Models without Batches

Atli Kosson, Vitaliy Chiley, Abhi Venigalla and
Joel Hestness, Urs Koster

Keywords Paper

0

0

0

0

18:00

26/04/2020

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius

Runtian Zhai, Chen Dan, Di He and
Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, Liwei Wang

Keywords Paper

Adversarial Robustness, Provable Adversarial Defense, Randomized Smoothing, Robustness Certification

0

0

0

0

5:10

06/12/2021

NAS-Bench-x11 and the Power of Learning Curves

Shen Yan, Colin White, Yash Savani, Frank Hutter

Keywords Paper

deep learning

0

0

0

0

14:03

19/08/2021

Topological Uncertainty: Monitoring Trained Neural Networks through Persistence of Activation Graphs

Théo Lacombe, Yuichi Ike, Mathieu Carrière and
Frédéric Chazal, Marc Glisse, Yuhei Umeda

Keywords Paper

Machine Learning, Deep Learning, Uncertainty Representations

0

0

0

0

14:35

06/12/2021

Speedy Performance Estimation for Neural Architecture Search

Robin Ru, Clare Lyle, Lisa Schut and
Miroslav Fil, Mark van der Wilk, Yarin Gal

Keywords Paper

deep learning

0

0

0

0

13:22

18/07/2021

Dense for the Price of Sparse: Improved Performance of Sparsely Initialized Networks via a Subspace Offset

Ilan Price, Jared Tanner

Keywords Paper

Deep Learning

0

0

0

0

5:42

12/07/2020

Overfitting in adversarially robust deep learning

Eric Wong, Leslie Rice, Zico Kolter

Keywords Paper

Adversarial Examples

0

0

0

0

14:44

18/07/2021

Weight-covariance alignment for adversarially robust neural networks

Panagiotis Eustratiadis, Henry Gouk, Da Li, Timothy Hospedales

Keywords Paper

Deep Learning, Adversarial Networks

0

0

0

0

4:46

06/12/2020

Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search

Houwen Peng, Hao Du, Hongyuan Yu and
QI LI, Jing Liao, Jianlong Fu

Keywords Paper

0

0

0

0

3:12

05/04/2021

Scaling Polyhedral Neural Network Verification on GPUs

Christoph Müller , François Serre, Gagandeep Singh and
Markus Püschel, Martin Vechev

Keywords Paper

0

0

0

0

22:07

05/04/2021

Scaling Polyhedral Neural Network Verification on GPUs

Christoph Müller , François Serre, Gagandeep Singh and
Markus Püschel, Martin Vechev

Keywords Paper

0

0

0

0

3:37

18/07/2021

Neural Architecture Search without Training

Joe Mellor, Jack Turner, Amos Storkey, Elliot Crowley

Keywords Paper

Deep Learning, Architectures

0

0

0

1

20:37

26/04/2020

Adversarially robust transfer learning

Ali Shafahi, Parsa Saadatpanah, Chen Zhu and
Amin Ghiasi, Christoph Studer, David Jacobs, Tom Goldstein

Keywords Paper

0

0

0

0

4:58

15/11/2020

Perfectly Parallel Fairness Certification of Neural Networks

Caterina Urban, Maria Christakis, Valentin Wüstholz, Fuyuan Zhang

Keywords Paper

Abstract Interpretation, Fairness, Static Analysis, Neural Networks

0

0

0

0

15:18

06/12/2020

Belief Propagation Neural Networks

Jonathan Kuck, Shuvam Chakraborty, Hao Tang and
Rachel Luo, Jiaming Song, Ashish Sabharwal, Stefano Ermon

Keywords Paper

0

0

0

0

3:22

18/07/2021

Learning Neural Network Subspaces

Mitchell Wortsman, Maxwell Horton, Carlos Guestrin and
Ali Farhadi, Mohammad Rastegari

Keywords Paper

Deep Learning, Applications, Dialog- or Communication-Based Learning, Algorithms, Representation Learning

0

0

0

0

5:07

08/12/2020

Dynamic Curriculum Learning for Low-Resource Neural Machine Translation

Chen Xu, Bojie Hu, Yufan Jiang and
Kai Feng, Zeyang Wang, Shen Huang, Qi Ju, Tong Xiao, Jingbo Zhu

Keywords Paper

0

0

0

0

13:28

22/11/2021

FFNB: Forgetting-Free Neural Blocks for Deep Continual Learning

Hichem Sahbi, Haoming Zhan

Keywords Paper

Continual and incremental learning, lifelong learning, catastrophic interference, catastrophic forgetting, dynamic neural networks, visual recognition

0

0

0

0

3:05

06/12/2021

Adversarial Robustness with Non-uniform Perturbations

Ecenaz Erdemir, Jeffrey Bickford, Luca Melis, Sergul Aydore

Keywords Paper

deep learning, optimization, machine learning, robustness, adversarial robustness and security

0

0

0

0

15:05

12/08/2020

SmartVerif: Push the Limit of Automation Capability of Verifying Security Protocols by Dynamic Strategies

Yan Xiong, Cheng Su, Wenchao Huang and
Fuyou Miao, Wansen Wang, Hengyi Ouyang

Keywords Paper

0

0

0

0

11:18

26/08/2020

Fenchel Lifted Networks: A Lagrange Relaxation of Neural Network Training

Fangda Gu, Armin Askari, Laurent El Ghaoui

Keywords Paper

0

0

0

0

14:27

14/06/2020

When NAS Meets Robustness: In Search of Robust Architectures Against Adversarial Attacks

Minghao Guo, Yuzhe Yang, Rui Xu and
Ziwei Liu, Dahua Lin

Keywords Paper

adversarial robustness, neural architecture search, adversarial examples, deep learning architectures, adversarial attacks

0

0

0

0

1:01

06/12/2020

Reparameterizing Mirror Descent as Gradient Descent

Ehsan Amid, Manfred K. Warmuth

Keywords Paper

0

0

0

0

3:34

26/04/2020

A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning

Soochan Lee, Junsoo Ha, Dongsu Zhang, Gunhee Kim

Keywords Paper

continual learning, task-free, task-agnostic

0

0

0

0

5:08

03/05/2021

Dataset Meta-Learning from Kernel Ridge-Regression

Timothy Nguyen, Zhourong Chen, Jaehoon Lee

Keywords Paper

dataset corruption, infinite-width networks, neural kernels, kernel-ridge regression, dataset compression, dataset distillation, meta-learning

0

0

0

0

4:59

06/12/2021

BulletTrain: Accelerating Robust Neural Network Training via Boundary Example Mining

Weizhe Hua, Yichi Zhang, Chuan Guo and
Zhiru Zhang, G. Edward Suh

Keywords Paper

deep learning, machine learning, robustness, adversarial robustness and security

0

0

0

0

6:36

12/07/2020

Adversarial Robustness via Runtime Masking and Cleansing

Yi-Hsuan Wu, Chia-Hung Yuan, Shan-Hung (Brandon) Wu

Keywords Paper

Adversarial Examples

0

0

0

0

13:38

30/11/2020

Towards Fast and Robust Adversarial Training for Image Classification

Erh-Chung Chen, Che-Rung Lee

Keywords Paper

0

0

0

0

8:42

19/08/2021

Verifying Reinforcement Learning up to Infinity

Edoardo Bacci, Mirco Giacobbe, David Parker

Keywords Paper

Machine Learning, Deep Reinforcement Learning, Validation and Verification, Learning in Robotics

0

0

0

0

14:57

14/06/2020

Fixed-Point Back-Propagation Training

Xishan Zhang, Shaoli Liu, Rui Zhang and
Chang Liu, Di Huang, Shiyi Zhou, Jiaming Guo, Qi Guo, Zidong Du, Tian Zhi, Yunji Chen

Keywords Paper

network quantization, fixed-point training, deep learning, neural network

1

0

0

0

1:01

06/12/2021

How Powerful are Performance Predictors in Neural Architecture Search?

Colin White, Arber Zela, Robin Ru and
Yang Liu, Frank Hutter

Keywords Paper

deep learning, optimization

0

0

0

0

15:02

14/06/2020

Learn2Perturb: An End-to-End Feature Perturbation Learning to Improve Adversarial Robustness

Ahmadreza Jeddi, Mohammad Javad Shafiee, Michelle Karg and
Christian Scharfenberger, Alexander Wong

Keywords Paper

adversarial robustness, network randomization, alternative back-propagation, trainable noise, adversarial training

0

0

0

0

1:01

19/04/2021

Bootstrapping relation extractors using syntactic search by examples

Matan Eyal, Asaf Amrami, Hillel Taub-Tabib, Yoav Goldberg

Keywords Paper

0

0

0

0

9:55

12/07/2020

Learning to Rank Learning Curves

Martin Wistuba, Tejaswini Pedapati

Keywords Paper

Transfer, Multitask and Meta-learning

0

0

0

0

15:34

26/04/2020

Picking Winning Tickets Before Training by Preserving Gradient Flow

Chaoqi Wang, Guodong Zhang, Roger Grosse

Keywords Paper

neural network, pruning before training, weight pruning

0

0

0

0

5:02