25/07/2020

Cascade model-based propensity estimation for counterfactual learning to rank

Ali Vardasbi, Maarten Rijke, Ilya Markov

Keywords: inverse propensity scoring, counterfactual learning to rank, click models

Abstract Paper Similar Papers

Abstract: Unbiased counterfactual learning to rank (CLTR) requires click propensities to compensate for the difference between user clicks and true relevance of search results via inverse propensity scoring (IPS). Current propensity estimation methods assume that user click behavior follows the position-based click model (PBM) and estimate click propensities based on this assumption. However, in reality, user clicks often follow the cascade model (CM), where users scan search results from top to bottom and where each next click depends on the previous one. In this cascade scenario, PBM-based estimates of propensities are not accurate, which, in turn, hurts CLTR performance. In this paper, we propose a propensity estimation method for the cascade scenario, called cascade model-based inverse propensity scoring (CM-IPS). We show that CM-IPS keeps CLTR performance close to the full-information performance in case the user clicks follow the CM, while PBM-based CLTR has a significant gap towards the full-information. The opposite is true if the user clicks follow PBM instead of the CM. Finally, we suggest a way to select between CM- and PBM-based propensity estimation methods based on historical user clicks.

The video of this talk cannot be embedded. You can watch it here:
https://dl.acm.org/doi/10.1145/3397271.3401299#sec-supp
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at SIGIR 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 11:32 
 0
 0
 0
 0
 2:06 
 0
 0
 0
 0
 2:51 
 0
 0
 0
 0
 2:19 
 0
 0
 0
 0
 6:39 
 0
 0
 0
 0
 12:25 
 0
 0
 0
 0
 13:18 
 0
 0
 0
 0
 6:03 
 0
 0
 0
 0
 17:01 
 0
 0
 0
 0
 9:07 
 0
 0
 0
 0
 2:07 
 0
 0
 0
 0
 10:59 
 0
 0
 0
 0
 9:21 
 0
 0
 0
 0
 18:21 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 14:21 
 0
 0
 0
 0
 2:38 
 0
 0
 0
 0
 16:41 
 0
 0
 0
 0
 9:01 
 0
 0
 0
 0
 13:53 
 0
 0
 0
 0
 2:54 
 0
 0
 0
 0
 11:21 
 0
 0
 0
 0
 2:57 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 2:59 
 0
 0
 0
 0
 12:45 
 0
 0
 0
 1
 15:55 
 0
 0
 0
 0
 2:06 
 0
 0
 0
 0
 14:13 
 0
 0
 0
 0
 13:03 
 0
 0
 0
 0
 18:26 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 15:06 
 0
 0
 0
 0
 13:57 
 0
 0
 0
 0
 13:43 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 18:32 
 0
 0
 0
 0
 2:35 
 0
 0
 0
 0
 8:51