12/07/2020

Logarithmic Regret for Online Control with Adversarial Noise

Dylan Foster, Max Simchowitz

Keywords: Reinforcement Learning - Theory

Abstract: We consider the problem of online control in a known linear dynamical system subject to adversarial noise. Existing regret bounds for this setting scale as $\sqrt{T}$ unless strong stochastic assumptions are imposed on the noise. We give the first algorithm with logarithmic regret for arbitrary adversarial noise sequences, provided that the state and control costs are given by fixed quadratic functions. We propose a novel analysis that combines a new variant of the performance difference lemma with techniques from optimal control, allowing us to reduce online control to online prediction with delayed feedback. Unlike prior work, which leverages the so-called online convex optimization with memory framework, our analysis \emph{does not} need to bound movement costs of the iterates, leading to logarithmic regret. Our performance difference lemma-based analysis may be of broader interest beyond linear control.

 0
 0
 0
 0
This is an embedded video. Talk and the respective paper are published at ICML 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd

Similar Papers