14/09/2020

Deep Learning, Grammar Transfer, and Transportation Theory

Kaixuan Zhang, Qinglong Wang, Lee Giles

Keywords: deep learning, grammar transfer, optimal transport, category theory

Abstract: Despite its widespread adoption and success, deep learning-based artificial intelligence is limited in providing an understandable decision-making process of what it does. This makes the “intelligence” part questionable since we expect real artificial intelligence to not only complete a given task but also perform in a way that is understandable. One way to approach this is to build a connection between artificial intelligence and human intelligence. Here, we use grammar transfer to demonstrate a paradigm that connects these two types of intelligence. Specifically, we define the action of transferring the knowledge learned by a recurrent neural network from one regular grammar to another grammar as grammar transfer. We are motivated by the theory that there is a natural correspondence between second-order recurrent neural networks and deterministic finite automata, which are uniquely associated with regular grammars. To study the process of grammar transfer, we propose a category based framework we denote as grammar transfer learning. Under this framework, we introduce three isomorphic categories and define ideal transfers by using transportation theory in operations research. By regarding the optimal transfer plan as a sensible operation from a human perspective, we then use it as a reference for examining whether a learning model behaves intelligently when performing the transfer task. Experiments under our framework demonstrate that this learning model can learn a grammar intelligently in general, but fails to follow the optimal way of learning.

 0
 0
 0
 0
This is an embedded video. Talk and the respective paper are published at ECML PKDD 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers