02/02/2021

Liquid Time-constant Networks

Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, Radu Grosu

Keywords:

Abstract Paper Similar Papers

Abstract: We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems modulated via nonlinear interlinked gates. The resulting models represent dynamical systems with varying (i.e., liquid) time-constants coupled to their hidden state, with outputs being computed by numerical differential equation solvers. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations, and give rise to improved performance on time-series prediction tasks. To demonstrate these properties, we first take a theoretical approach to find bounds over their dynamics, and compute their expressive power by the trajectory length measure in a latent trajectory space. We then conduct a series of time-series prediction experiments to manifest the approximation capability of Liquid Time-Constant Networks (LTCs) compared to classical and modern RNNs.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949141
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 14:46 
 0
 0
 0
 0
 12:32 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 5:26 
 0
 0
 0
 0
 3:50 
 0
 0
 0
 0
 15:38 
 0
 0
 0
 0
 18:32 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 5:37 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 2:56 
 0
 0
 0
 0
 18:52 
 0
 0
 0
 0
 8:56 
 0
 0
 0
 0
 5:22 
 0
 0
 0
 0
 9:10 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 14:59 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 7:57 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 5:31 
 0
 0
 0
 0
 15:01 
 0
 0
 0
 0
 12:09 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 4:56 
 0
 0
 0
 0
 15:10 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 13:29 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 6:00 
 0
 0
 0
 0
 5:25