25/07/2020

Distributed equivalent substitution training for large-scale recommender systems

Haidong Rong, Yangzihao Wang, Feihu Zhou, Junjie Zhai, Haiyang Wu, Rui Lan, Fan Li, Han Zhang, Yuekui Yang, Zhenyu Guo, Di Wang

Keywords: ranking systems, synchronous training, recommender systems, dynamic sparse features

Abstract Paper Similar Papers

Abstract: We present Distributed Equivalent Substitution (DES) training, a novel distributed training framework for large-scale recommender systems with dynamic sparse features. DES introduces fully synchronous training to large-scale recommendation system for the first time by reducing communication, thus making the training of commercial recommender systems converge faster and reach better CTR. DES requires much less communication by substituting the weights-rich operators with the computationally equivalent sub-operators and aggregating partial results instead of transmitting the huge sparse weights directly through the network. Due to the use of synchronous training on large-scale Deep Learning Recommendation Models (DLRMs), DES achieves higher AUC(Area Under ROC). We successfully apply DES training on multiple popular DLRMs of industrial scenarios. Experiments show that our implementation outperforms the state-of-the-art PS-based training framework, achieving up to 68.7

The video of this talk cannot be embedded. You can watch it here:
https://dl.acm.org/doi/10.1145/3397271.3401113#sec-supp
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at SIGIR 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 16:52 
 0
 0
 0
 0
 9:59 
 0
 0
 0
 0
 16:57 
 0
 0
 0
 0
 4:45 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:22 
 0
 0
 0
 0
 4:36 
 0
 0
 0
 1
 3:14 
 0
 0
 0
 0
 20:07 
 0
 0
 0
 0
 4:47 
 0
 0
 0
 0
 10:47 
 0
 0
 0
 0
 2:59 
 0
 0
 0
 0
 8:48 
 0
 0
 0
 0
 15:59 
 0
 0
 0
 0
 5:06 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 3:29 
 0
 0
 0
 0
 22:38 
 0
 0
 0
 1
 6:22 
 0
 0
 0
 0
 11:29 
 0
 0
 0
 0
 5:02 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 14:47 
 0
 0
 0
 0
 6:26 
 0
 0
 0
 0
 8:30 
 0
 0
 0
 0
 0:56 
 0
 0
 0
 0
 8:45 
 0
 0
 0
 0
 5:06 
 0
 0
 0
 0
 5:01 
 0
 0
 0
 0
 1:01 
 0
 0
 0
 0
 3:12 
 0
 0
 0
 0
 5:18 
 0
 0
 0
 0
 5:16 
 0
 0
 0
 0
 5:13 
 0
 0
 0
 0
 3:00 
 0
 0
 0
 0
 13:02 
 0
 0
 0
 0
 13:06 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 4:58 
 0
 0
 0
 0
 15:00