02/02/2021

Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks

Xiyue Zhang, Xiaoning Du, Xiaofei Xie, Lei Ma, Yang Liu, Meng Sun

Keywords:

Abstract Paper Similar Papers

Abstract: Recurrent Neural Networks (RNNs) have demonstrated their effectiveness in learning and processing sequential data (e.g., speech and natural language). However, due to the black-box nature of neural networks, understanding the decision logic of RNNs is quite challenging. Some recent progress has been made to approximate the behavior of an RNN by weighted automata. They provide better interpretability, but still suffer from poor scalability. In this paper, we propose a novel approach to extracting weighted automata with the guidance of a target RNN's decision and context information. In particular, we identify the patterns of RNN's step-wise predictive decisions to instruct the formation of automata states. Further, we propose a state composition method to enhance the context-awareness of the extracted model. Our in-depth evaluations on typical RNN tasks, including language model and classification, demonstrate the effectiveness and advantage of our method over the state-of-the-arts. The evaluation results show that our method can achieve accurate approximation of an RNN even on large-scale tasks.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948527
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 13:17 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 12:23 
 0
 0
 0
 0
 14:45 
 0
 0
 0
 0
 14:49 
 0
 0
 0
 0
 11:54 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 16:25 
 0
 0
 0
 0
 3:00 
 0
 0
 0
 0
 17:06 
 0
 0
 0
 0
 14:55 
 0
 0
 0
 0
 6:05 
 0
 0
 0
 0
 11:02 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 11:40 
 0
 0
 0
 0
 16:26 
 0
 0
 0
 0
 11:18 
 0
 0
 0
 0
 17:37 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 14:33 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 8:17 
 0
 0
 0
 0
 11:55 
 0
 0
 0
 0
 12:13 
 0
 0
 0
 0
 8:38 
 0
 0
 0
 0
 14:35 
 0
 0
 0
 1
 9:29 
 0
 0
 0
 0
 18:02 
 0
 0
 0
 0
 11:50 
 0
 0
 0
 0
 14:45 
 0
 0
 0
 0
 14:31 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 6:14 
 0
 0
 0
 0
 14:26 
 0
 0
 0
 0
 10:53 
 0
 0
 0
 0
 20:16 
 0
 0
 0
 0
 11:24 
 0
 0
 0
 0
 14:07