06/12/2021

Framing RNN as a kernel method: A neural ODE approach

Adeline Fermanian, Pierre Marion, Jean-Philippe Vert, Gérard Biau

Keywords: theory, deep learning, kernel methods

Abstract Paper Similar Papers

Abstract: Building on the interpretation of a recurrent neural network (RNN) as a continuous-time neural differential equation, we show, under appropriate conditions, that the solution of a RNN can be viewed as a linear function of a specific feature set of the input sequence, known as the signature. This connection allows us to frame a RNN as a kernel method in a suitable reproducing kernel Hilbert space. As a consequence, we obtain theoretical guarantees on generalization and stability for a large class of recurrent networks. Our results are illustrated on simulated datasets.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at NeurIPS 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 16:02 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 13:29 
 0
 0
 0
 0
 4:48 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 10:15 
 0
 0
 0
 0
 12:32 
 0
 0
 0
 0
 3:04 
 0
 0
 0
 0
 18:48 
 0
 0
 0
 0
 5:20 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 3:07 
 0
 0
 0
 0
 14:55 
 0
 0
 0
 0
 2:56 
 0
 0
 0
 0
 5:06 
 0
 0
 0
 0
 4:48 
 0
 0
 0
 0
 15:38 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 3:10 
 0
 0
 0
 0
 15:28 
 0
 0
 0
 0
 3:06 
 0
 0
 0
 0
 6:00 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 18:32 
 0
 0
 0
 0
 5:34 
 0
 0
 0
 0
 15:56 
 0
 0
 0
 0
 14:26 
 0
 0
 0
 0
 2:31 
 0
 0
 0
 0
 15:08 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 12:38 
 0
 0
 0
 0
 5:12