26/08/2020

Momentum in Reinforcement Learning

Nino Vieillard, Bruno Scherrer, Olivier Pietquin, Matthieu Geist

Keywords:

Abstract Paper Similar Papers

Abstract: We adapt the optimization's concept of momentum to reinforcement learning. Seeing the state-action value functions as an anlog to the gradients in optimization, we interpret momentum as an average of consecutive $q$-functions. We derive Momentum Value Iteration (MoVI), a variation of Value iteration that incorporates this momentum idea. Our analysis shows that this allows MoVI to average errors over successive iterations. We show that the proposed approach can be readily extended to deep learning. Specifically,we propose a simple improvement on DQN based on MoVI, and experiment it on Atari games.

 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AISTATS 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 20:06 
 1
 0
 0
 0
 18:06 
 0
 0
 0
 0
 2:53 
 0
 0
 0
 0
 4:52 
 0
 0
 0
 0
 2:44 
 0
 0
 0
 0
 2:30 
 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 15:50 
 0
 0
 0
 0
 5:04 
 0
 0
 0
 0
 13:59 
 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 15:45 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 6:51 
 0
 0
 0
 0
 5:16 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 8:39 
 0
 0
 0
 0
 14:00 
 0
 0
 0
 0
 5:17 
 0
 0
 0
 0
 5:05 
 0
 0
 0
 0
 18:09 
 0
 0
 0
 0
 15:12 
 0
 0
 0
 0
 15:05 
 1
 0
 0
 1
 14:38 
 0
 0
 0
 0
 4:38 
 0
 0
 0
 0
 19:21 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 5:30 
 0
 0
 0
 0
 21:04 
 0
 0
 0
 0
 10:49 
 0
 0
 0
 0
 14:06 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 9:48 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 5:16 
 0
 0
 0
 0
 12:57 
 0
 0
 0
 0
 15:04