02/02/2021

Differentially Private and Communication Efficient Collaborative Learning

Jiahao Ding, Guannan Liang, Jinbo Bi, Miao Pan

Keywords:

Abstract Paper Similar Papers

Abstract: Collaborative learning has received huge interests due to its capability of exploiting the collective computing power of the wireless edge devices. However, during the learning process, model updates using local private samples and large-scale parameter exchanges among agents impose severe privacy concerns and communication bottleneck. In this paper, to address these problems, we propose two differentially private (DP) and communication efficient algorithms, called Q-DPSGD-1 and Q-DPSGD-2. In Q-DPSGD-1, each agent first performs local model updates by a DP gradient descent method to provide the DP guarantee and then quantizes the local model before transmitting it to neighbors to improve communication efficiency. In Q-DPSGD-2, each agent injects discrete Gaussian noise to enforce DP guarantee after first quantizing the local model. Moreover, we track the privacy loss of both approaches under the Renyi DP and provide convergence analysis for both convex and non-convex loss functions. The proposed methods are evaluated in extensive experiments on real-world datasets and the empirical results validate our theoretical findings.

The video of this talk cannot be embedded. You can watch it here:
https://slideslive.com/38947989
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 2:43 
 0
 0
 0
 1
 15:50 
 0
 0
 0
 0
 3:19 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 2:47 
 0
 0
 0
 0
 5:04 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 3:08 
 0
 0
 0
 0
 3:16 
 0
 0
 0
 0
 15:04 
 0
 0
 0
 0
 15:44 
 0
 0
 0
 0
 17:07 
 0
 0
 0
 0
 14:59 
 0
 0
 0
 0
 4:44 
 0
 0
 0
 0
 3:13 
 0
 0
 0
 0
 3:08 
 0
 0
 0
 1
 5:12 
 0
 0
 0
 0
 14:30 
 0
 0
 0
 0
 12:09 
 0
 0
 0
 2
 4:57 
 0
 0
 0
 0
 5:14 
 0
 0
 0
 0
 5:21 
 0
 0
 0
 0
 5:51 
 0
 0
 0
 0
 13:35 
 0
 0
 0
 0
 12:55 
 0
 0
 0
 0
 14:17 
 0
 0
 0
 0
 19:56 
 0
 0
 0
 0
 14:03 
 0
 0
 0
 0
 14:59 
 1
 1
 0
 1
 5:09 
 0
 0
 0
 0
 18:24 
 0
 0
 0
 0
 13:44 
 1
 1
 0
 0
 14:46 
 0
 0
 0
 1
 4:41 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 1
 12:45 
 0
 0
 0
 2
 11:39 
 0
 0
 0
 0
 5:00 
 0
 0
 0
 0
 18:22