02/02/2021

Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimate

Mirco Mutti, Lorenzo Pratissoli, Marcello Restelli

Keywords:

Abstract Paper Similar Papers

Abstract: In a reward-free environment, what is a suitable intrinsic objective for an agent to pursue so that it can learn an optimal task-agnostic exploration policy? In this paper, we argue that the entropy of the state distribution induced by finite-horizon trajectories is a sensible target. Especially, we present a novel and practical policy-search algorithm, Maximum Entropy POLicy optimization (MEPOL), to learn a policy that maximizes a non-parametric, $k$-nearest neighbors estimate of the state distribution entropy. In contrast to known methods, MEPOL is completely model-free as it requires neither to estimate the state distribution of any policy nor to model transition dynamics. Then, we empirically show that MEPOL allows learning a maximum-entropy exploration policy in high-dimensional, continuous-control domains, and how this policy facilitates learning meaningful reward-based tasks downstream.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949218
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:07 
 0
 0
 0
 0
 16:03 
 0
 0
 0
 0
 11:19 
 0
 0
 0
 0
 14:52 
 0
 0
 0
 0
 12:08 
 0
 0
 0
 0
 13:47 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 13:11 
 0
 0
 0
 0
 2:21 
 0
 0
 0
 0
 12:57 
 0
 0
 0
 0
 21:51 
 0
 0
 0
 0
 17:08 
 0
 0
 0
 0
 16:54 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 4:31 
 0
 0
 0
 0
 10:12 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 15:07 
 0
 0
 0
 0
 14:35 
 0
 0
 0
 0
 14:37 
 0
 0
 0
 0
 4:54 
 0
 0
 0
 0
 16:28 
 0
 0
 0
 0
 19:23 
 0
 0
 0
 0
 4:49 
 0
 0
 0
 0
 14:40 
 0
 0
 0
 0
 14:29 
 0
 0
 0
 0
 5:15 
 0
 0
 0
 0
 11:53 
 0
 0
 0
 0
 13:09 
 0
 0
 0
 0
 14:14 
 0
 0
 0
 0
 12:55 
 0
 0
 0
 0
 15:18 
 0
 0
 0
 0
 15:07 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 13:24 
 0
 0
 0
 0
 10:21 
 0
 0
 0
 0
 8:57 
 0
 0
 0
 0
 14:57 
 0
 0
 0
 0
 15:16 
 0
 0
 0
 0
 5:15