02/02/2021

An Efficient Algorithm for Deep Stochastic Contextual Bandits

Tan Zhu, Guannan Liang, Chunjiang Zhu, Haining Li, Jinbo Bi

Keywords:

Abstract Paper Similar Papers

Abstract: In stochastic contextual bandit (SCB) problems, an agent selects an action based on certain observed context to maximize the cumulative reward over iterations. Recently there have been a few studies using a deep neural network (DNN) to predict the expected reward for an action, and the DNN is trained by a stochastic gradient based method. However, convergence analysis has been greatly ignored to examine whether and where these methods converge. In this work, we formulate the SCB that uses a DNN reward function as a non-convex stochastic optimization problem, and design a stage-wised stochastic gradient descent algorithm to optimize the problem and determine the action policy. We prove that with high probability, the action sequence chosen by our algorithm converges to a greedy action policy respecting a local optimal reward function. Extensive experiments have been performed to demonstrate the effectiveness and efficiency of the proposed algorithm on multiple real-world datasets.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38948120
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 15:12 
 0
 0
 0
 0
 4:10 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 17:13 
 0
 0
 0
 0
 17:04 
 0
 0
 0
 0
 14:17 
 0
 0
 0
 0
 21:51 
 0
 0
 0
 0
 12:55 
 0
 0
 0
 0
 14:40 
 0
 0
 0
 0
 14:49 
 0
 0
 0
 0
 3:07 
 0
 0
 0
 0
 11:20 
 0
 0
 0
 0
 11:33 
 0
 0
 0
 0
 13:08 
 0
 0
 0
 0
 14:46 
 0
 0
 0
 0
 4:35 
 0
 0
 0
 0
 3:11 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 3:04 
 0
 0
 0
 0
 12:11 
 0
 0
 0
 0
 2:43 
 0
 0
 0
 0
 3:11 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 5:53 
 0
 0
 0
 0
 15:07 
 0
 0
 0
 0
 14:17 
 0
 0
 0
 0
 11:47 
 0
 0
 0
 0
 3:05 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 3:02 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 17:58 
 0
 0
 0
 0
 14:41 
 0
 0
 0
 0
 13:04 
 0
 0
 0
 0
 10:19 
 0
 0
 0
 0
 18:34 
 0
 0
 0
 0
 3:42 
 0
 0
 0
 0
 3:09 
 0
 0
 0
 0
 11:34