02/02/2021

Learning to Reweight Imaginary Transitions for Model-Based Reinforcement Learning

Wenzhen Huang, Qiyue Yin, Junge Zhang, Kaiqi Huang

Keywords:

Abstract Paper Similar Papers

Abstract: Model-based reinforcement learning (RL) is more sample efficient than model-free RL by using imaginary trajectories generated by the learned dynamics model. When the model is inaccurate or biased, imaginary trajectories may be deleterious for training the action-value and policy functions. To alleviate such problem, this paper proposes to adaptively reweight the imaginary transitions, so as to reduce the negative effects of poorly generated trajectories. More specifically, we evaluate the effect of an imaginary transition by calculating the change of the loss computed on the real samples when we use the transition to train the action-value and policy functions. Based on this evaluation criterion, we construct the idea of reweighting each imaginary transition by a well-designed meta-gradient algorithm. Extensive experimental results demonstrate that our method outperforms state-of-the-art model-based and model-free RL algorithms on multiple tasks. Visualization of our changing weights further validates the necessity of utilizing reweight scheme.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948081
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 15:31 
 0
 0
 0
 0
 12:35 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 5:16 
 0
 0
 0
 0
 12:53 
 0
 0
 0
 0
 13:11 
 0
 0
 0
 0
 11:19 
 0
 0
 0
 0
 5:16 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 4:12 
 0
 0
 0
 0
 12:17 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 5:06 
 0
 0
 0
 0
 12:24 
 0
 0
 0
 0
 10:52 
 0
 0
 0
 0
 14:07 
 0
 0
 0
 1
 3:24 
 0
 0
 0
 0
 19:12 
 0
 0
 0
 0
 5:43 
 0
 0
 0
 0
 8:47 
 0
 0
 0
 0
 3:31 
 0
 0
 0
 0
 4:49 
 0
 0
 0
 0
 3:09 
 0
 0
 0
 0
 5:09 
 1
 0
 0
 0
 10:44 
 0
 0
 0
 0
 13:21 
 0
 0
 0
 0
 14:35 
 1
 1
 0
 0
 13:17 
 0
 0
 0
 0
 14:20 
 0
 0
 0
 0
 4:32 
 0
 0
 0
 0
 11:42 
 0
 0
 0
 0
 3:34 
 0
 0
 0
 0
 10:46 
 0
 0
 0
 0
 13:36 
 1
 0
 0
 0
 10:31 
 0
 0
 0
 0
 15:17 
 0
 0
 0
 0
 13:51 
 0
 0
 0
 0
 14:34 
 0
 0
 0
 0
 14:35