02/02/2021

WCSAC: Worst-Case Soft Actor Critic for Safety-Constrained Reinforcement Learning

Qisong Yang, Thiago D. Simão, Simon H Tindemans, Matthijs T. J. Spaan

Keywords:

Abstract Paper Similar Papers

Abstract: Safe exploration is regarded as a key priority area for reinforcement learning research. With separate reward and safety signals, it is natural to cast it as constrained reinforcement learning, where expected long-term costs of policies are constrained. However, it can be hazardous to set constraints on the expected safety signal without considering the tail of the distribution. For instance, in safety-critical domains, worst-case analysis is required to avoid disastrous results. We present a novel reinforcement learning algorithm called Worst-Case Soft Actor Critic, which extends the Soft Actor Critic algorithm with a safety critic to achieve risk control. More specifically, a certain level of conditional Value-at-Risk from the distribution is regarded as a safety measure to judge the constraint satisfaction, which guides the change of adaptive safety weights to achieve a trade-off between reward and safety. As a result, we can optimize policies under the premise that their worst-case performance satisfies the constraints. The empirical analysis shows that our algorithm attains better risk control compared to expectation-based methods.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948598
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:07 
 0
 0
 0
 0
 13:26 
 2
 1
 0
 0
 6:50 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 10:40 
 0
 0
 0
 0
 2:51 
 0
 0
 0
 0
 14:05 
 0
 0
 0
 0
 5:09 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 3:06 
 1
 0
 0
 0
 13:54 
 0
 0
 0
 0
 3:15 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 5:20 
 0
 0
 0
 0
 3:15 
 0
 0
 0
 0
 15:00 
 0
 0
 0
 0
 10:52 
 0
 0
 0
 0
 3:09 
 0
 0
 0
 0
 14:39 
 0
 0
 0
 0
 5:03 
 0
 0
 0
 0
 2:41 
 0
 0
 0
 0
 14:14 
 0
 0
 0
 0
 5:19 
 0
 0
 0
 0
 13:49 
 0
 0
 0
 0
 16:03 
 0
 0
 0
 0
 2:21 
 0
 0
 0
 0
 12:27 
 0
 0
 0
 0
 10:03 
 0
 0
 0
 0
 3:09 
 0
 0
 0
 0
 12:49 
 0
 0
 0
 0
 12:35 
 0
 0
 0
 0
 13:40 
 0
 0
 0
 0
 18:07 
 0
 0
 0
 0
 5:19 
 0
 0
 0
 0
 13:32 
 0
 0
 0
 0
 16:28 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 8:45 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 0
 14:52