Deep Residual Reinforcement Learning (Extended Abstract)

Abstract: We revisit residual algorithms in both model-free and model-based reinforcement learning settings. We propose the bidirectional target network technique to stabilize residual algorithms, yielding a residual version of DDPG that significantly outperforms vanilla DDPG in commonly used benchmarks. Moreover, we find the residual algorithm an effective approach to the distribution mismatch problem in model-based planning. Compared with the existing TD(k) method, our residual-based method makes weaker assumptions about the model and yields a greater performance boost.

06/12/2020

Deep Residual Reinforcement Learning (Extended Abstract)

Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson

Comments

Similar Papers

On the Convergence of Smooth Regularized Approximate Value Iteration Schemes

Elena Smirnova, Elvis Dohmatob

Keywords Abstract Paper

Logistic q-learning

Joan Bas-Serrano, Sebastian Curi, Andreas Krause, Gergely Neu

Keywords Abstract Paper

From Label Smoothing to Label Relaxation

Julian Lienen, Eyke Hüllermeier

Keywords Abstract Paper

Discount Factor as a Regularizer in Reinforcement Learning

Ron Amit, Kamil Ciosek, Ron Meir

Keywords Abstract Paper

Reinforcement Learning - General

DriftSurf: Stable-State / Reactive-State Learning under Concept Drift

Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, Phil Gibbons

Keywords Abstract Paper

Algorithms, Online Learning Algorithms

Learning Randomly Perturbed Structured Predictors for Direct Loss Minimization

Hedda Cohen Indelman, Tamir Hazan

Keywords Abstract Paper

Algorithms, Structured Prediction, Algorithms, Collaborative Filtering, Applications, Recommender Systems

Offline Reinforcement Learning with Fisher Divergence Critic Regularization

Ilya Kostrikov, Rob Fergus, Jonathan Tompson, Ofir Nachum

Keywords Abstract Paper

Reinforcement Learning and Planning, Deep RL

Settling the Variance of Multi-Agent Policy Gradients

Jakub Grudzien Kuba, Muning Wen, Linghui Meng and shangding gu, Haifeng Zhang, David Mguni, Jun Wang, Yaodong Yang

Keywords Abstract Paper

deep learning, reinforcement learning and planning

CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks

Sakshi Varshney, Vinay Kumar Verma, P. K. Srijith and Lawrence Carin, Piyush Rai

Keywords Abstract Paper

generative model, representation learning, continual learning

Learning to Reweight Imaginary Transitions for Model-Based Reinforcement Learning

Wenzhen Huang, Qiyue Yin, Junge Zhang, Kaiqi Huang

Keywords Abstract Paper

Asynchronous Decentralized Optimization With Implicit Stochastic Variance Reduction

Kenta Niwa, Guoqiang Zhang, W. Bastiaan Kleijn and Noboru Harada, Hiroshi Sawada, Akinori Fujino

Keywords Abstract Paper

Optimization, Distributed and Parallel Optimization

Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift

Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, Geoffrey Gordon

Keywords Abstract Paper

Scalable Differential Privacy with Certified Robustness in Adversarial Learning

Hai Phan, My T. Thai, Han Hu and Ruoming Jin, Tong Sun, Dejing Dou

Keywords Abstract Paper

Privacy-preserving Statistics and Machine Learning

No-regret Exploration in Contextual Reinforcement Learning

Aditya Modi, Ambuj Tewari

Keywords Abstract Paper

Almost Optimal Model-Free Reinforcement Learningvia Reference-Advantage Decomposition

Zihan Zhang, Yuan Zhou, Xiangyang Ji

Keywords Abstract Paper

Adaptive Discretization for Model-Based Reinforcement Learning

Sean Sinclair, Tianyu Wang, Gauri Jain and Sid Banerjee, Christina Yu

Keywords Abstract Paper

Ranking Policy Gradient

Kaixiang Lin, Jiayu Zhou

Keywords Abstract Paper

Sample-efficient reinforcement learning, off-policy learning.

Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation

Xinjie Fan, Yizhe Zhang, Zhendong Wang, Mingyuan Zhou

Keywords Abstract Paper

binary softmax, discrete variables, policy gradient, pseudo actions, reinforcement learning, variance reduction

Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation

Bo Pang, Zhong-Ping Jiang

Keywords Abstract Paper

Responsive Safety in Reinforcement Learning

Adam Stooke, Joshua Achiam, Pieter Abbeel

Keywords Abstract Paper

Reinforcement Learning - Deep RL

A New Representation of Successor Features for Transfer across Dissimilar Environments

Majid Abdolshah, Hung Le, Thommen Karimpanal George and Sunil Gupta, Santu Rana, Svetha Venkatesh

Keywords Abstract Paper

Reinforcement Learning and Planning

COMBO: Conservative Offline Model-Based Policy Optimization

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Jakub Grudzien Kuba, Muning Wen, Linghui Meng and
shangding gu, Haifeng Zhang, David Mguni, Jun Wang, Yaodong Yang

Keywords Paper

Sakshi Varshney, Vinay Kumar Verma, P. K. Srijith and
Lawrence Carin, Piyush Rai

Keywords Paper

Keywords Paper

Kenta Niwa, Guoqiang Zhang, W. Bastiaan Kleijn and
Noboru Harada, Hiroshi Sawada, Akinori Fujino

Keywords Paper

Keywords Paper

Hai Phan, My T. Thai, Han Hu and
Ruoming Jin, Tong Sun, Dejing Dou

Keywords Paper

Keywords Paper

Keywords Paper

Sean Sinclair, Tianyu Wang, Gauri Jain and
Sid Banerjee, Christina Yu

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Majid Abdolshah, Hung Le, Thommen Karimpanal George and
Sunil Gupta, Santu Rana, Svetha Venkatesh

Keywords Paper

Tianhe Yu, Aviral Kumar, Rafael Rafailov and
Aravind Rajeswaran, Sergey Levine, Chelsea Finn

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Trung Dang, Om Thakkar, Swaroop Ramaswamy and
Rajiv Mathews, Peter Chin, Françoise Beaufays

Keywords Paper

Hanbo Zhang, Site Bai, Xuguang Lan and
David Hsu, Nanning Zheng

Keywords Paper

Keywords Paper

Keywords Paper

Keywords Paper

Fei Yu, Mo Zhang, Hexin Dong and
Sheng Hu, Bin Dong, Li Zhang

Keywords Paper

Giancarlo Kerg, bhargav104 Kanuparthi, Anirudh Goyal ALIAS PARTH GOYAL and
Kyle Goyette, Yoshua Bengio, Guillaume Lajoie

Keywords Paper

Keywords Paper

Arushi Jain, Gandharv Patil, Ayush Jain and
Khimya Khetarpal, Doina Precup

Keywords Paper