02/02/2021

Disentangled Information Bottleneck

Ziqi Pan, Li Niu, Jianfu Zhang, Liqing Zhang

Keywords:

Abstract Paper Similar Papers

Abstract: The information bottleneck (IB) method is a technique for extracting information that is relevant for predicting the target random variable from the source random variable, which is typically implemented by optimizing the IB Lagrangian that balances the compression and prediction terms. However, the IB Lagrangian is hard to optimize, and multiple trials for tuning values of Lagrangian multiplier are required. Moreover, we show that the prediction performance strictly decreases as the compression gets stronger during optimizing the IB Lagrangian. In this paper, we implement the IB method from the perspective of supervised disentangling. Specifically, we introduce Disentangled Information Bottleneck (DisenIB) that is consistent on compressing source maximally without target prediction performance loss (maximum compression). Theoretical and experimental results demonstrate that our method is consistent on maximum compression, and performs well in terms of generalization, robustness to adversarial attack, out-of-distribution detection, and supervised disentangling.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38947930
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 5:20 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 5:16 
 1
 0
 0
 0
 13:50 
 0
 0
 0
 0
 4:23 
 0
 0
 0
 0
 20:14 
 0
 0
 0
 0
 12:24 
 0
 0
 0
 0
 10:16 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 5:25 
 0
 0
 0
 0
 13:27 
 0
 0
 0
 0
 14:02 
 0
 0
 0
 0
 5:13 
 0
 0
 0
 0
 11:41 
 0
 0
 0
 0
 3:06 
 0
 0
 0
 0
 14:43 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 18:27 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 3:03 
 0
 0
 0
 0
 5:36 
 0
 0
 0
 0
 3:08 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 3:00 
 0
 0
 0
 0
 12:35 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 14:05 
 0
 0
 0
 0
 11:16 
 0
 0
 0
 0
 12:41 
 0
 0
 0
 0
 9:02 
 0
 0
 0
 0
 11:26 
 0
 0
 0
 0
 9:42 
 0
 0
 0
 0
 3:26 
 0
 0
 0
 0
 6:23 
 0
 0
 0
 0
 5:13 
 0
 0
 0
 0
 19:29 
 0
 0
 0
 0
 4:23