19/08/2021

Information-Theoretic Methods in Deep Neural Networks: Recent Advances and Emerging Opportunities

Shujian Yu, Luis Sanchez Giraldo, Jose Principe

Keywords: Machine learning, General

Abstract Paper Similar Papers

Abstract: We present a review on the recent advances and emerging opportunities around the theme of analyzing deep neural networks (DNNs) with information-theoretic methods. We first discuss popular information-theoretic quantities and their estimators. We then introduce recent developments on information-theoretic learning principles (e.g., loss functions, regularizers and objectives) and their parameterization with DNNs. We finally briefly review current usages of information-theoretic concepts in a few modern machine learning problems and list a few emerging opportunities.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at IJCAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 14:26 
 0
 0
 0
 0
 14:11 
 0
 0
 0
 0
 16:02 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 14:28 
 0
 0
 0
 0
 3:01 
 0
 0
 0
 0
 15:59 
 0
 0
 0
 0
 5:50 
 0
 0
 0
 0
 11:32 
 0
 0
 0
 0
 8:19 
 0
 0
 0
 0
 13:11 
 0
 0
 0
 0
 5:17 
 0
 0
 0
 0
 7:04 
 0
 0
 0
 0
 12:32 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 12:00 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 17:10 
 0
 0
 0
 0
 14:44 
 0
 0
 0
 0
 11:44 
 0
 0
 0
 0
 14:14 
 0
 0
 0
 0
 15:38 
 0
 0
 0
 0
 2:19 
 0
 0
 0
 0
 5:35 
 0
 0
 0
 0
 3:15 
 0
 0
 0
 1
 9:57 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 8:56 
 0
 0
 0
 0
 9:38 
 0
 0
 0
 0
 0:59 
 0
 0
 0
 0
 14:23 
 0
 0
 0
 0
 4:53 
 0
 0
 0
 1
 14:04 
 0
 0
 0
 0
 13:13 
 0
 0
 0
 0
 13:45 
 1
 0
 0
 0
 2:43 
 0
 0
 0
 0
 5:26 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 4:50 
 0
 0
 0
 0
 3:10