02/02/2021

TempLe: Learning Template of Transitions for Sample Efficient Multi-task RL

Yanchao Sun, Xiangyu Yin, Furong Huang

Keywords:

Abstract Paper Similar Papers

Abstract: Transferring knowledge among various environments is important for efficiently learning multiple tasks online. Most existing methods directly use the previously learned models or previously learned optimal policies to learn new tasks. However, these methods may be inefficient when the underlying models or optimal policies are substantially different across tasks. In this paper, we propose Template Learning (TempLe), a PAC-MDP method for multi-task reinforcement learning that could be applied to tasks with varying state/action space without prior knowledge of inter-task mappings. TempLe gains sample efficiency by extracting similarities of the transition dynamics across tasks even when their underlying models or optimal policies have limited commonalities. We present two algorithms for an ``online'' and a ``finite-model'' setting respectively. We prove that our proposed TempLe algorithms achieve much lower sample complexity than single-task learners or state-of-the-art multi-task methods. We show via systematically designed experiments that our TempLe method universally outperforms the state-of-the-art multi-task methods (PAC-MDP or not) in various settings and regimes.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38949227
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:32 
 0
 0
 0
 0
 4:48 
 0
 0
 0
 0
 3:27 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 11:14 
 0
 0
 0
 0
 4:57 
 0
 0
 0
 0
 7:26 
 0
 0
 0
 0
 13:59 
 0
 0
 0
 0
 15:18 
 0
 0
 0
 0
 11:12 
 0
 0
 0
 0
 13:34 
 0
 0
 0
 0
 13:43 
 0
 0
 0
 0
 5:31 
 0
 0
 0
 0
 2:59 
 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 5:26 
 0
 0
 0
 0
 10:41 
 0
 0
 0
 0
 7:41 
 0
 0
 0
 0
 3:54 
 0
 0
 0
 0
 6:01 
 0
 0
 0
 0
 0:50 
 0
 1
 0
 0
 3:22 
 0
 0
 0
 0
 15:07 
 0
 0
 0
 0
 14:50 
 0
 0
 0
 0
 3:21 
 0
 0
 0
 0
 11:59 
 0
 0
 0
 0
 11:27 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 1
 6:22 
 0
 0
 0
 0
 4:49 
 1
 0
 0
 0
 5:08 
 0
 0
 0
 0
 19:14 
 0
 0
 0
 0
 14:16 
 0
 0
 0
 0
 3:07 
 0
 0
 0
 0
 14:26 
 0
 0
 0
 0
 5:22 
 0
 0
 0
 0
 5:13 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 3:11 
 0
 0
 0
 0
 5:04