02/02/2021

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

Yaru Hao, Li Dong, Furu Wei, Ke Xu

Keywords:

Abstract Paper Similar Papers

Abstract: The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attribute model decisions to individual input features with different saliency measures, but they fail to explain how these input features interact with each other to reach predictions. In this paper, we propose a self-attention attribution method to interpret the information interactions inside Transformer. We take BERT as an example to conduct extensive studies. Firstly, we apply self-attention attribution to identify the important attention heads, while others can be pruned with marginal performance degradation. Furthermore, we extract the most salient dependencies in each layer to construct an attribution tree, which reveals the hierarchical interactions inside Transformer. Finally, we show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949312
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 19:21 
 0
 0
 0
 0
 14:29 
 0
 0
 0
 0
 14:48 
 0
 0
 0
 0
 7:10 
 0
 0
 0
 0
 13:49 
 0
 0
 0
 0
 11:55 
 0
 0
 0
 0
 14:30 
 0
 0
 0
 0
 5:34 
 0
 0
 0
 0
 15:46 
 0
 0
 0
 0
 2:52 
 0
 0
 0
 0
 4:20 
 0
 0
 0
 0
 13:09 
 0
 0
 0
 0
 11:57 
 0
 0
 0
 0
 14:14 
 0
 0
 0
 0
 12:02 
 0
 0
 0
 0
 2:47 
 0
 0
 0
 0
 4:51 
 0
 0
 0
 0
 18:50 
 0
 0
 0
 0
 17:56 
 0
 0
 0
 0
 21:30 
 0
 0
 0
 0
 12:51 
 0
 0
 0
 0
 16:14 
 0
 0
 0
 1
 5:13 
 0
 0
 0
 0
 19:30 
 0
 0
 0
 0
 11:12 
 0
 0
 0
 0
 14:05 
 0
 0
 0
 0
 10:23 
 0
 0
 0
 0
 4:33 
 0
 0
 0
 0
 3:11 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 4:48 
 0
 0
 0
 0
 19:21 
 0
 0
 0
 0
 5:12 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 9:52 
 0
 0
 0
 0
 7:46 
 0
 0
 0
 0
 20:08 
 0
 0
 0
 0
 11:41 
 0
 0
 0
 0
 5:08 
 0
 0
 0
 0
 13:20