06/12/2020

Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs

Talgat Daulbaev, Alexandr Katrutsa, Larisa Markeeva, Julia Gusak, Andrzej Cichocki, Ivan Oseledets

Keywords:

Abstract Paper Similar Papers

Abstract: We propose a simple interpolation-based method for the efficient approximation of gradients in neural ODE models. We compare it with reverse dynamic method (known in literature as “adjoint method”) to train neural ODEs on classification, density estimation and inference approximation tasks. We also propose a theoretical justification of our approach using logarithmic norm formalism. As a result, our method allows faster model training than the reverse dynamic method what was confirmed and validated by extensive numerical experiments for several standard benchmarks.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at NeurIPS 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 14:46 
 0
 0
 0
 0
 15:02 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 12:09 
 0
 0
 0
 0
 3:26 
 0
 0
 1
 1
 11:26 
 0
 0
 0
 0
 8:59 
 0
 0
 0
 0
 8:47 
 0
 0
 0
 0
 2:59 
 0
 0
 0
 0
 12:51 
 0
 0
 0
 0
 5:29 
 0
 0
 0
 0
 17:43 
 0
 0
 0
 0
 12:15 
 0
 0
 0
 0
 14:35 
 0
 0
 0
 0
 14:04 
 0
 0
 0
 0
 9:58 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 15:12 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 2:56 
 0
 0
 0
 0
 15:56 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 15:26 
 1
 0
 0
 0
 14:15 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 14:59 
 0
 0
 0
 0
 5:25 
 0
 0
 0
 0
 4:08 
 0
 0
 0
 0
 18:32 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 5:38 
 0
 0
 0
 0
 6:00 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 8:56 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 14:56