06/12/2021

Information-theoretic generalization bounds for black-box learning algorithms

Hrayr Harutyunyan, Maxim Raginsky, Greg Ver Steeg, Aram Galstyan

Keywords: theory, deep learning

Abstract Paper Similar Papers

Abstract: We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm. These bounds improve over the existing information-theoretic bounds, are applicable to a wider range of algorithms, and solve two key challenges: (a) they give meaningful results for deterministic algorithms and (b) they are significantly easier to estimate. We show experimentally that the proposed bounds closely follow the generalization gap in practical scenarios for deep learning.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at NeurIPS 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 5:18 
 0
 1
 0
 0
 5:08 
 2
 1
 0
 0
 13:27 
 0
 0
 0
 0
 14:26 
 0
 0
 0
 0
 2:38 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 9:01 
 0
 0
 0
 0
 12:31 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 3:01 
 0
 0
 0
 0
 3:54 
 0
 0
 0
 0
 15:04 
 0
 0
 0
 0
 16:12 
 0
 0
 0
 0
 15:26 
 0
 0
 0
 0
 7:18 
 0
 0
 0
 0
 7:39 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 14:37 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 5:17 
 0
 0
 0
 0
 13:41 
 0
 0
 0
 1
 4:50 
 0
 0
 0
 0
 13:51 
 0
 0
 0
 0
 4:25 
 0
 0
 0
 0
 8:20 
 0
 0
 0
 0
 13:23 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 7:41 
 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 4:46 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 3:00 
 1
 0
 0
 1
 12:23 
 0
 0
 0
 0
 14:29