Relative Variational Intrinsic Control

02/02/2021

Relative Variational Intrinsic Control

Kate Baumli, David Warde-Farley, Steven Hansen, Volodymyr Mnih

Keywords:

Abstract Paper Similar Papers

Abstract: In the absence of external rewards, agents can still learn useful behaviors by identifying and mastering a set of diverse skills within their environment. Existing skill learning methods use mutual information objectives to incentivize each skill to be diverse and distinguishable from the rest. However, if care is not taken to constrain the ways in which the skills are diverse, trivially diverse skill sets can arise. To ensure useful skill diversity, we propose a novel skill learning objective, Relative Variational Intrinsic Control (RVIC), which incentivizes learning skills that are distinguishable in how they change the agent's relationship to its environment. The resulting set of skills tiles the space of affordances available to the agent. We qualitatively analyze skill behaviors on multiple environments and show how RVIC skills are more useful than skills discovered by existing methods in hierarchical reinforcement learning.

The video of this talk cannot be embedded. You can watch it here:

https://slideslive.com/38948125

(Link will open in new window)

0

0

0

0

Share

This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment

no comments yet

Similar Papers

06/12/2020

One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL

Saurabh Kumar, Aviral Kumar, Sergey Levine, Chelsea Finn

Keywords Paper

0

0

0

0

3:24

12/07/2020

Meta Variance Transfer: Learning to Augment from the Others

Seong-Jin Park, Seungju Han, Ji-won Baek and
Insoo Kim, Juhwan Song, Hae Beom Lee, Jae-Joon Han, Sung Ju Hwang

Keywords Paper

Transfer, Multitask and Meta-learning

0

0

0

0

14:59

06/12/2021

Visual Adversarial Imitation Learning using Variational Models

Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, Chelsea Finn

Keywords Paper

theory, reinforcement learning and planning, adversarial robustness and security, representation learning

0

0

0

0

7:25

06/12/2021

Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning

Jinxin Liu, Hao Shen, Donglin Wang and
Yachen Kang, Qiangxing Tian

Keywords Paper

reinforcement learning and planning, domain adaptation

0

0

0

0

8:07

03/05/2021

HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworks

Zhou Xian, Shamit Lal, Hsiao-Yu Tung and
Anthony Platanios, Katerina Fragkiadaki

Keywords Paper

0

0

0

0

5:46

06/12/2021

An Efficient Transfer Learning Framework for Multiagent Reinforcement Learning

Tianpei Yang, Weixun Wang, Hongyao Tang and
Jianye Hao, Zhaopeng Meng, Hangyu Mao, Dong Li, Wulong Liu, Yingfeng Chen, Yujing Hu, Changjie Fan, Chengwei Zhang

Keywords Paper

reinforcement learning and planning, transfer learning

0

0

0

0

15:21

26/04/2020

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

Roberta Raileanu, Tim Rocktäschel

Keywords Paper

reinforcement learning, exploration, curiosity

0

0

0

0

4:48

18/07/2021

Imitation by Predicting Observations

Andrew Jaegle, Yury Sulsky, Arun Ahuja and
Jake Bruce, Rob Fergus, Greg Wayne

Keywords Paper

Reinforcement Learning and Planning, Others

0

0

0

0

5:15

06/12/2021

Learning Markov State Abstractions for Deep Reinforcement Learning

Cameron Allen, Neev Parikh, Omer Gottesman, George Konidaris

Keywords Paper

reinforcement learning and planning, contrastive learning, representation learning

0

0

0

0

12:31

18/07/2021

A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning

Dong Ki Kim, Miao Liu, Matthew Riemer and
Chuangchuang Sun, Marwa Abdulhai, Golnaz Habibi, Sebastian Lopez-Cot, Gerald Tesauro, Jonathan How

Keywords Paper

Reinforcement Learning and Planning, Multi-Agent RL, Algorithms, Representation Learning, Algorithms, Relational Learning

0

0

0

0

5:20

03/05/2021

OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning

Anurag Ajay, Aviral Kumar, Pulkit Agrawal and
Sergey Levine, Ofir Nachum

Keywords Paper

Unsupervised Learning, Offline Reinforcement Learning, Primitive Discovery

0

0

0

0

5:08

03/05/2021

Recurrent Independent Mechanisms

Anirudh Goyal, Alex Lamb, Jordan Hoffmann and
Shagun Sodhani, Sergey Levine, Yoshua Bengio, Bernhard Schoelkopf

Keywords Paper

better generalization, modular representations, learning mechanisms

0

0

0

0

10:14

03/05/2021

Domain-Robust Visual Imitation Learning with Mutual Information Constraints

Edoardo Cetin, Oya Celiktutan

Keywords Paper

Domain Adaption, Third-Person Imitation, Observational Imitation, Reinforcement Learning, Machine Learning, Mutual Information, Imitation Learning

0

0

0

0

4:51

06/12/2021

Agent Modelling under Partial Observability for Deep Reinforcement Learning

Georgios Papoudakis, Filippos Christianos, Stefano Albrecht

Keywords Paper

reinforcement learning and planning

0

0

0

0

13:22

19/08/2021

Don’t Do What Doesn’t Matter: Intrinsic Motivation with Action Usefulness

Mathieu Seurin, Florian Strub, Philippe Preux, Olivier Pietquin

Keywords Paper

Machine Learning, Reinforcement Learning, Deep Reinforcement Learning

0

0

0

0

14:48

18/07/2021

Learning Routines for Effective Off-Policy Reinforcement Learning

Edoardo Cetin, Oya Celiktutan

Keywords Paper

Reinforcement Learning and Planning, Deep RL

0

0

0

0

5:17

06/12/2020

Robust Pre-Training by Adversarial Contrastive Learning

Ziyu Jiang, Tianlong Chen, Ting Chen, Zhangyang Wang

Keywords Paper

0

0

0

0

3:26

06/12/2021

Variational Multi-Task Learning with Gumbel-Softmax Priors

Jiayi Shen, Xiantong Zhen, Marcel Worring, Ling Shao

Keywords Paper

machine learning, generative model

0

0

0

0

13:09

12/07/2020

Weakly-Supervised Disentanglement Without Compromises

Francesco Locatello, Ben Poole, Gunnar Raetsch and
Bernhard Schölkopf, Olivier Bachem, Michael Tschannen

Keywords Paper

Representation Learning

0

0

0

0

14:47

06/12/2021

On Calibration and Out-of-Domain Generalization

Yoav Wald, Amir Feder, Daniel Greenfeld, Uri Shalit

Keywords Paper

machine learning, domain adaptation, causality

0

0

0

0

11:00

16/11/2020

Attention-Privileged Reinforcement Learning

Sasha Salter, Dushyant Rao, Markus Wulfmeier and
Raia Hadsell, Ingmar Posner

Keywords Paper

0

0

0

0

4:54

03/05/2021

Learning What To Do by Simulating the Past

David Lindner, Rohin Shah, Pieter Abbeel, Anca Dragan

Keywords Paper

reinforcement learning, imitation learning, reward learning

0

0

0

0

5:09

06/12/2020

Cooperative Heterogeneous Deep Reinforcement Learning

Han Zheng, Pengfei Wei, Jing Jiang and
Guodong Long, Qinghua Lu, Chengqi Zhang

Keywords Paper

0

0

0

0

3:08

22/11/2021

Rich Semantics Improve Few-Shot Learning

Mohamed Afham Mohamed Aflal, Salman Khan, Muhammad Haris Khan and
Muzammal Naseer, Fahad Shahbaz Khan

Keywords Paper

few shot learning, multimodal learning, transformers in vision

0

0

0

0

2:47

03/05/2021

Task-Agnostic Morphology Evolution

Donald Hejna III, Pieter Abbeel, Lerrel Pinto

Keywords Paper

evolution, morphology, empowerment, unsupervised, information theory

0

0

0

0

3:59

03/05/2021

Fast And Slow Learning Of Recurrent Independent Mechanisms

Kanika Madan, Nan Rosemary Ke, Anirudh Goyal and
Bernhard Schoelkopf, Yoshua Bengio

Keywords Paper

better generalization, modular representations, learning mechanisms

0

0

0

0

5:09

06/12/2021

Discovery of Options via Meta-Learned Subgoals

Vivek Veeriah, Tom Zahavy, Matteo Hessel and
Zhongwen Xu, Junhyuk Oh, Iurii Kemaev, Hado van Hasselt, David Silver, Satinder Singh

Keywords Paper

deep learning, reinforcement learning and planning

0

0

0

0

4:13

06/12/2021

Conservative Data Sharing for Multi-Task Offline Reinforcement Learning

Tianhe Yu, Aviral Kumar, Yevgen Chebotar and
Karol Hausman, Sergey Levine, Chelsea Finn

Keywords Paper

reinforcement learning and planning

0

0

0

0

14:27

18/07/2021

UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learning

Tarun Gupta, Anuj Mahajan, Bei Peng and
Wendelin Boehmer, Shimon Whiteson

Keywords Paper

Reinforcement Learning and Planning, Multi-Agent RL

0

0

0

0

5:23

18/07/2021

MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration

Jin Zhang, Jianhao Wang, Hao Hu and
Tong Chen, Yingfeng Chen, Changjie Fan, Chongjie Zhang

Keywords Paper

Algorithms, Multitask, Transfer, and Meta Learning

0

0

0

0

4:19

26/04/2020

Scalable and Order-robust Continual Learning with Additive Parameter Decomposition

Jaehong Yoon, Saehoon Kim, Eunho Yang, Sung Ju Hwang

Keywords Paper

Continual Learning, Lifelong Learning, Catastrophic Forgetting, Deep Learning

0

0

0

0

5:07

03/05/2021

Regularized Inverse Reinforcement Learning

Wonseok Jeon, Chen-Yang Su, Paul Barde and
Thang Doan, Derek Nowrouzezahrai, Joelle Pineau

Keywords Paper

reinforcement learning, regularized markov decision processes, reward learning, inverse reinforcement learning

0

0

0

0

9:50

06/12/2020

Effective Diversity in Population Based Reinforcement Learning

Jack Parker-Holder, Aldo Pacchiano, Krzysztof M Choromanski, Stephen J Roberts

Keywords Paper

0

0

0

0

3:23

06/12/2020

What shapes feature representations? Exploring datasets, architectures, and training

Katherine L. Hermann, Andrew Lampinen

Keywords Paper

0

0

0

0

2:36

02/02/2021

Latent Independent Excitation for Generalizable Sensor-based Cross-Person Activity Recognition

Hangwei Qian, Sinno Jialin Pan, Chunyan Miao

Keywords Paper

0

0

0

0

16:06

26/04/2020

Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents

Christian Rupprecht, Cyril Ibrahim, Christopher J. Pal

Keywords Paper

Visualization, Reinforcement Learning, Safety

0

0

0

0

4:52

03/05/2021

Learning from others' mistakes: Avoiding dataset biases without modeling them

Victor Sanh, Thomas Wolf, Yonatan Belinkov, Alexander M Rush

Keywords Paper

product of experts, dataset bias, natural language processing

0

0

0

0

5:14

03/05/2021

Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning

Dong Bok Lee, Dongchan Min, Seanie Lee, Sung Ju Hwang

Keywords Paper

Unsupervised Learning, Variational Autoencoders, Unsupervised Meta-learning, Meta-Learning

0

0

0

0

13:31

06/12/2021

Towards Robust Bisimulation Metric Learning

Mete Kemertas, Tristan Aumentado-Armstrong

Keywords Paper

reinforcement learning and planning, robustness, representation learning

0

0

0

0

12:24

14/09/2020

Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning

Arjun Manoharan, Rahul Ramesh, Balaraman Ravindran

Keywords Paper

hierarchical reinforcement learning, policy distillation

0

0

0

0

13:49