23/08/2020

On sampling top-k recommendation evaluation

Dong Li, Ruoming Jin, Jing Gao, Zhi Liu

Keywords: recommender systems, hit ratio, evaluation metric, top-k, recall

Abstract Paper Similar Papers

Abstract: Recently, Rendle has warned that the use of sampling-based top-k metrics might not suffice. This throws a number of recent studies on deep learning-based recommendation algorithms, and classic non-deep-learning algorithms using such a metric, into jeopardy. In this work, we thoroughly investigate the relationship between the sampling and global top-K Hit-Ratio (HR, or Recall), originally proposed by Koren[2] and extensively used by others. By formulating the problem of aligning sampling top-k (SHR@k) and global top-K (HR@K) Hit-Ratios through a mapping function f, so that SHR@k  HR@f(k), we demonstrate both theoretically and experimentally that the sampling top-k Hit-Ratio provides an accurate approximation of its global (exact) counterpart, and can consistently predict the correct winners (the same as indicate by their corresponding global Hit-Ratios).

The video of this talk cannot be embedded. You can watch it here:
https://dl.acm.org/doi/10.1145/3394486.3403262#sec-supp
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at KDD 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 9:55 
 0
 0
 0
 0
 14:58 
 0
 0
 0
 0
 16:11 
 0
 0
 0
 0
 15:32 
 0
 0
 0
 0
 17:24 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 1:03 
 0
 0
 0
 0
 14:51 
 0
 0
 0
 0
 13:48 
 0
 0
 0
 0
 11:50 
 0
 0
 0
 0
 9:21 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 10:41 
 0
 0
 0
 0
 10:53 
 1
 0
 0
 0
 13:50 
 0
 0
 0
 0
 11:28 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 2:57 
 0
 0
 0
 0
 10:46 
 0
 0
 0
 0
 4:23 
 0
 0
 0
 0
 15:34 
 0
 0
 0
 0
 14:09 
 0
 0
 0
 0
 14:21 
 0
 0
 0
 0
 9:26 
 0
 0
 0
 0
 11:45 
 0
 0
 0
 0
 3:26 
 0
 0
 0
 0
 14:20 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 2:56 
 0
 0
 0
 0
 14:07 
 0
 0
 0
 0
 5:36 
 0
 0
 0
 0
 15:25 
 0
 0
 0
 0
 3:14 
 0
 0
 0
 0
 9:59 
 0
 0
 0
 0
 4:07 
 0
 0
 0
 0
 3:03 
 0
 0
 0
 0
 15:17 
 0
 0
 0
 0
 5:11 
 0
 0
 0
 0
 15:22 
 0
 0
 0
 0
 15:40