02/02/2021

Learning with Safety Constraints: Sample Complexity of Reinforcement Learning for Constrained MDPs

Aria HasanzadeZonuzy, Archana Bura, Dileep Kalathil, Srinivas Shakkottai

Keywords:

Abstract Paper Similar Papers

Abstract: Many physical systems have underlying safety considerations that require that the policy employed ensures the satisfaction of a set of constraints. The analytical formulation usually takes the form of a Constrained Markov Decision Process (CMDP). We focus on the case where the CMDP is unknown, and RL algorithms obtain samples to discover the model and compute an optimal constrained policy. Our goal is to characterize the relationship between safety constraints and the number of samples needed to ensure a desired level of accuracy---both objective maximization and constraint satisfaction---in a PAC sense. We explore two classes of RL algorithms, namely, (i) a generative model based approach, wherein samples are taken initially to estimate a model, and (ii) an online approach, wherein the model is updated as samples are obtained. Our main finding is that compared to the best known bounds of the unconstrained regime, the sample complexity of constrained RL algorithms are increased by a factor that is logarithmic in the number of constraints, which suggests that the approach may be easily utilized in real systems.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948993
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 13:26 
 0
 0
 0
 0
 5:20 
 0
 0
 0
 0
 4:50 
 0
 0
 0
 0
 15:25 
 0
 0
 0
 0
 5:15 
 0
 0
 0
 0
 12:49 
 0
 0
 0
 0
 19:33 
 0
 0
 0
 0
 3:15 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 14:18 
 0
 0
 0
 0
 13:58 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 18:23 
 0
 0
 0
 0
 13:31 
 0
 0
 0
 0
 14:40 
 0
 0
 0
 0
 3:29 
 0
 0
 0
 0
 15:16 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 19:14 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 12:27 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 10:08 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 3:08 
 0
 0
 0
 0
 10:40 
 0
 0
 0
 0
 16:19 
 0
 0
 0
 0
 12:47 
 0
 0
 0
 0
 13:40 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 11:21 
 0
 0
 0
 0
 3:25 
 0
 0
 0
 0
 3:07 
 0
 0
 0
 0
 4:59 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 11:07 
 0
 0
 0
 0
 13:54 
 0
 0
 0
 0
 8:45 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 5:15