Inverse Reinforcement Learning From Like-Minded Teachers

02/02/2021

Inverse Reinforcement Learning From Like-Minded Teachers

Ritesh Noothigattu, Tom Yan, Ariel D. Procaccia

Keywords:

Abstract Paper Similar Papers

Abstract: We study the problem of learning a policy in a Markov decision process (MDP) based on observations of the actions taken by multiple teachers. We assume that the teachers are like-minded in that their reward functions -- while different from each other -- are random perturbations of an underlying reward function. Under this assumption, we demonstrate that inverse reinforcement learning algorithms that satisfy a certain property -- that of matching feature expectations -- yield policies that are approximately optimal with respect to the underlying reward function, and that no algorithm can do better in the worst case. We also show how to efficiently recover the optimal policy when the MDP has one state -- a setting that is akin to multi-armed bandits.

The video of this talk cannot be embedded. You can watch it here:

https://slideslive.com/38948713

(Link will open in new window)

0

0

0

0

Share

This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment

no comments yet

Similar Papers

03/08/2020

What You See May Not Be What You Get: UCB Bandit Algorithms Robust to $\varepsilon$-Contamination

Laura Niss, Ambuj Tewari

Keywords Paper

0

0

0

0

8:02

06/12/2020

Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space

Shangchen Du, Shan You, Xiaojie Li and
Jianlong Wu, Fei Wang, Chen Qian, Changshui Zhang

Keywords Paper

0

0

0

0

3:22

06/12/2021

Adversarial Intrinsic Motivation for Reinforcement Learning

Ishan Durugkar, Mauricio Tec, Scott Niekum, Peter Stone

Keywords Paper

reinforcement learning and planning, generative model

0

0

0

0

13:11

06/12/2020

Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher

Guangda Ji, Zhanxing Zhu

Keywords Paper

0

0

0

0

3:19

20/07/2020

Large deviations for the perceptron model and consequences for active learning

Hugo Cui, Luca Saglietti, Lenka Zdeborova

Keywords Paper

0

0

0

0

14:19

06/12/2021

Online learning in MDPs with linear function approximation and bandit feedback.

Gergely Neu, Julia Olkhovskaya

Keywords Paper

reinforcement learning and planning, bandits, online learning

0

0

0

0

13:24

12/07/2020

Meta-learning with Stochastic Linear Bandits

Leonardo Cella, Alessandro Lazaric, Massimiliano Pontil

Keywords Paper

Transfer, Multitask and Meta-learning

1

1

0

0

13:17

22/11/2021

Self-supervised Knowledge Distillation for Few-shot Learning

Jathushan Rajasegaran, Salman Khan, Munawar Hayat and
Fahad Shahbaz Khan, Mubarak Shah

Keywords Paper

Self-supervision, Knowledge Distillation, Few-shot Learning

0

0

0

0

2:49

06/12/2021

Learning One Representation to Optimize All Rewards

Ahmed Touati, Yann Ollivier

Keywords Paper

deep learning, reinforcement learning and planning, representation learning

0

0

0

0

14:52

18/07/2021

Adaptive Sampling for Best Policy Identification in Markov Decision Processes

Aymen Al Marjani, Alexandre Proutiere

Keywords Paper

Theory, RL, Decisions and Control Theory

0

0

0

0

5:35

03/05/2021

Meta-learning with negative learning rates

Alberto Bernacchia

Keywords Paper

Meta-learning

0

0

0

0

5:19

06/12/2021

Credal Self-Supervised Learning

Julian Lienen, Eyke Hüllermeier

Keywords Paper

self-supervised learning, vision, semi-supervised learning

0

0

0

0

15:01

06/12/2021

Training Over-parameterized Models with Non-decomposable Objectives

Harikrishna Narasimhan, Aditya Menon

Keywords Paper

optimization, machine learning, fairness

0

0

0

0

8:28

06/12/2021

Comprehensive Knowledge Distillation with Causal Intervention

Xiang Deng, Zhongfei Zhang

Keywords Paper

representation learning, causality

0

0

0

0

12:24

06/12/2021

Active clustering for labeling training data

Quentin Lutz, Elie de Panafieu, Maya Stein, Alex Scott

Keywords Paper

clustering, active learning

0

0

0

0

10:42

03/05/2021

Rethinking Soft Labels for Knowledge Distillation: A Bias–Variance Tradeoff Perspective

Helong Zhou, Liangchen Song, Jiajie Chen and
Ye Zhou, Guoli Wang, Junsong Yuan, Qian Zhang

Keywords Paper

teacher-student model, soft labels, Knowledge distillation

0

0

0

0

2:20

06/12/2021

Exponential Separation between Two Learning Models and Adversarial Robustness

Grzegorz Gluch, Ruediger Urbanke

Keywords Paper

theory, robustness, adversarial robustness and security

0

0

0

0

15:11

13/04/2021

Learning with risk-averse feedback under potentially heavy tails

Matthew Holland, El Mehdi Haress

Keywords Paper

0

0

0

0

2:44

02/02/2021

Reinforcement Learning with Trajectory Feedback

Yonathan Efroni, Nadav Merlis, Shie Mannor

Keywords Paper

0

0

0

0

14:17

06/12/2021

Online Selective Classification with Limited Feedback

Aditya Gangrade, Anil Kag, Ashok Cutkosky, Venkatesh Saligrama

Keywords Paper

machine learning, online learning

0

0

0

0

15:14

18/07/2021

Learning from Similarity-Confidence Data

Yuzhou Cao, Lei Feng, Yitian Xu and
Bo An, Gang Niu, Masashi Sugiyama

Keywords Paper

Algorithms, Semi-Supervised Learning

0

0

0

0

4:05

12/07/2020

Time-Consistent Self-Supervision for Semi-Supervised Learning

Tianyi Zhou, Shengjie Wang, Jeff Bilmes

Keywords Paper

Unsupervised and Semi-Supervised Learning

0

0

0

0

14:37

02/02/2021

Stable Adversarial Learning under Distributional Shifts

Jiashuo Liu, Zheyan Shen, Peng Cui and
Linjun Zhou, Kun Kuang, Bo Li, Yishi Lin

Keywords Paper

0

0

0

0

14:30

06/12/2021

Regime Switching Bandits

Xiang Zhou, Yi Xiong, Ningyuan Chen, Xuefeng GAO

Keywords Paper

reinforcement learning and planning, bandits, online learning

0

0

0

0

13:47

06/12/2021

Learning curves of generic features maps for realistic datasets with a teacher-student model

Bruno Loureiro, Cedric Gerbelot, Hugo Cui and
Sebastian Goldt, Florent Krzakala, Marc Mezard, Lenka Zdeborová

Keywords Paper

deep learning, machine learning, kernel methods

0

0

0

0

12:59

14/06/2020

Revisiting Knowledge Distillation via Label Smoothing Regularization

Li Yuan, Francis EH Tay, Guilin Li and
Tao Wang, Jiashi Feng

Keywords Paper

knowledge distillation, label smoothing regularization

0

0

0

0

4:21

18/07/2021

Accelerating Safe Reinforcement Learning with Constraint-mismatched Baseline Policies

Jimmy Yang, Justinian Rosca, Karthik Narasimhan, Peter Ramadge

Keywords Paper

Algorithms, Adversarial Learning, Applications, Computer Vision; Deep Learning, Adversarial Networks; Deep Learning, Generative Models, Reinforcement Learning and Planning, Deep RL

0

0

0

0

5:20

18/07/2021

Continual Learning in the Teacher-Student Setup: Impact of Task Similarity

Sebastian Lee, Sebastian Goldt, Andrew Saxe

Keywords Paper

Theory, Models of Learning and Generalization

0

0

0

0

5:36

06/12/2020

Agnostic Learning with Multiple Objectives

Corinna Cortes, Mehryar Mohri, Javier Gonzalvo, Dmitry Storcheus

Keywords Paper

0

0

0

0

3:07

02/02/2021

PULNS: Positive-Unlabeled Learning with Effective Negative Sample Selector

Chuan Luo, Pu Zhao, Chen Chen and
Bo Qiao, Chao Du, Hongyu Zhang, Wei Wu, Shaowei Cai, Bing He, Saravanakumar Rajmohan, Qingwei Lin

Keywords Paper

0

0

0

0

18:08

19/08/2021

Hierarchical Class-Based Curriculum Loss

Palash Goyal, Divya Choudhary, Shalini Ghosh

Keywords Paper

Machine Learning, Classification, Multi-instance; Multi-label; Multi-view learning, Deep Learning

0

0

0

0

14:31

06/12/2020

Mix and Match: An Optimistic Tree-Search Approach for Learning Models from Mixture Distributions

Matthew Faw, Rajat Sen, Karthikeyan Shanmugam and
Constantine Caramanis, Sanjay Shakkottai

Keywords Paper

0

0

0

0

3:24

04/08/2021

Online Markov Decision Processes with Aggregate Bandit Feedback

Alon Cohen, Haim Kaplan, Tomer Koren, Yishay Mansour

Keywords Paper

0

0

0

0

13:07

12/07/2020

Learning the Valuations of a $k$-demand Agent

Hanrui Zhang, Vincent Conitzer

Keywords Paper

Learning Theory

0

0

0

0

14:51

06/12/2021

Accommodating Picky Customers: Regret Bound and Exploration Complexity for Multi-Objective Reinforcement Learning

Jingfeng Wu, Vladimir Braverman, Lin Yang

Keywords Paper

theory, reinforcement learning and planning

0

0

0

0

14:22

18/07/2021

A Distribution-dependent Analysis of Meta Learning

Mikhail Konobeev, Ilja Kuzborskij, Csaba Szepesvari

Keywords Paper

Theory, Statistical Learning Theory

0

0

0

0

5:06

12/07/2020

On the Sample Complexity of Adversarial Multi-Source PAC Learning

Nikola Konstantinov, Elias Frantar, Dan Alistarh, Christoph H. Lampert

Keywords Paper

Trustworthy Machine Learning

0

0

0

0

15:05

06/12/2020

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Stefano Sarao Mannelli, Eric Vanden-Eijnden, Lenka Zdeborová

Keywords Paper

Algorithms -> Model Selection and Structure Learning; Algorithms -> Representation Learning; Theory -> Computational Complexity, Reinforcement Learning and Planning -> Markov Decision Processes

0

0

0

0

3:22

12/07/2020

Learning with Good Feature Representations in Bandits and in RL with a Generative Model

Gellért Weisz, Tor Lattimore, Csaba Szepesvari

Keywords Paper

Reinforcement Learning - Theory

0

0

0

0

15:20

30/11/2020

DiscFace: Minimum Discrepancy Learning for Deep Face Recognition

Insoo Kim, Seungju Han, Seong-Jin Park and
Ji-won Baek, Jinwoo Shin, Jae-Joon Han, Changkyu Choi

Keywords Paper

0

0

0

0

8:21