Personalized Cross-Silo Federated Learning on Non-IID Data

02/02/2021

Personalized Cross-Silo Federated Learning on Non-IID Data

Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, Yong Zhang

Keywords:

Abstract Paper Similar Papers

Abstract: Non-IID data present a tough challenge for federated learning. In this paper, we explore a novel idea of facilitating pairwise collaborations between clients with similar data. We propose FedAMP, a new method employing federated attentive message passing to facilitate similar clients to collaborate more. We establish the convergence of FedAMP for both convex and non-convex models, and propose a heuristic method to further improve the performance of FedAMP when clients adopt deep neural networks as personalized models. Our extensive experiments on benchmark data sets demonstrate the superior performance of the proposed methods.

The video of this talk cannot be embedded. You can watch it here:

https://slideslive.com/38948676

(Link will open in new window)

0

0

0

0

Share

This is an embedded video. Talk and the respective paper are published at AAAI 2021 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment

no comments yet

Similar Papers

18/07/2021

Federated Continual Learning with Weighted Inter-client Transfer

Jaehong Yoon, Wonyong Jeong, GiWoong Lee and
Eunho Yang, Sung Ju Hwang

Keywords Paper

Algorithms, Continual Learning

0

0

0

1

4:41

06/12/2021

Parameterized Knowledge Transfer for Personalized Federated Learning

Jie Zhang, Song Guo, Xiaosong Ma and
Haozhao Wang, Wenchao Xu, Feijie Wu

Keywords Paper

optimization, federated learning

0

0

0

0

7:09

25/07/2020

Transfer learning via contextual invariants for one-to-many cross-domain recommendation

Adit Krishnan, Mahashweta Das, Mangesh Bendre and
Hao Yang, Hari Sundaram

Keywords Paper

data sparsity, transfer learning, cross-domain recommendation, contextual invariants, neural layer adaptation

0

0

0

0

19:24

18/07/2021

Debiasing Model Updates for Improving Personalized Federated Training

Durmus Alp Emre Acar, Yue Zhao, Ruizhao Zhu and
Ramon Matas, Matthew Mattina, Paul Whatmough, Venkatesh Saligrama

Keywords Paper

Algorithms

0

0

0

2

7:24

26/04/2020

Sharing Knowledge in Multi-Task Deep Reinforcement Learning

Carlo D'Eramo, Davide Tateo, Andrea Bonarini and
Marcello Restelli, Jan Peters

Keywords Paper

Deep Reinforcement Learning, Multi-Task

0

0

0

0

4:27

18/07/2021

Exploiting Shared Representations for Personalized Federated Learning

Liam Collins, Hamed Hassani, Aryan Mokhtari, Sanjay Shakkottai

Keywords Paper

Algorithms, Multitask, Transfer, and Meta Learning

1

1

0

1

5:09

03/05/2021

Adaptive Federated Optimization

Sashank Reddi, Zachary Charles, Manzil Zaheer and
Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, Brendan McMahan

Keywords Paper

distributed optimization, optimization, adaptive optimization, Federated learning

1

1

0

0

5:11

12/07/2020

Communication-Efficient Federated Learning with Sketching

Daniel Rothchild, Ashwinee Panda, Enayat Ullah and
Nikita Ivkin, Vladimir Braverman, Joseph Gonzalez, Ion Stoica, Raman Arora

Keywords Paper

Optimization - Large Scale, Parallel and Distributed

0

0

0

1

15:26

06/12/2021

Celebrating Diversity in Shared Multi-Agent Reinforcement Learning

Li Chenghao, Tonghan Wang, Chengjie Wu and
Qianchuan Zhao, Jun Yang, Chongjie Zhang

Keywords Paper

deep learning, optimization, reinforcement learning and planning

0

0

0

0

6:00

18/07/2021

Cross-Gradient Aggregation for Decentralized Learning from Non-IID Data

Yasaman Esfandiari, Sin Yong Tan, Zhanhong Jiang and
Aditya Balu, Ethan Herron, Chinmay Hegde, Soumik Sarkar

Keywords Paper

Optimization, Distributed and Parallel Optimization

1

0

0

1

5:15

18/07/2021

Quasi-global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Tao Lin, Praneeth Karimireddy, Sebastian Stich, Martin Jaggi

Keywords Paper

Deep Learning, Optimization for Deep Networks

0

0

0

0

5:14

18/07/2021

Personalized Federated Learning using Hypernetworks

Aviv Shamsian, Aviv Navon, Ethan Fetaya, Gal Chechik

Keywords Paper

Algorithms, Multitask, Transfer, and Meta Learning

1

1

0

1

5:06

06/12/2021

Gradient Driven Rewards to Guarantee Fairness in Collaborative Machine Learning

Xinyi Xu, Lingjuan Lyu, Xingjun Ma and
Chenglin Miao, Chuan Sheng Foo, Bryan Kian Hsiang Low

Keywords Paper

machine learning, fairness, federated learning

0

0

0

0

15:03

06/12/2021

Federated Reconstruction: Partially Local Federated Learning

Karan Singhal, Hakim Sidahmed, Zachary Garrett and
Shanshan Wu, John Rush, Sushant Prakash

Keywords Paper

robustness, meta learning, privacy, federated learning

0

0

0

2

11:39

06/12/2020

An Efficient Framework for Clustered Federated Learning

Avishek Ghosh, Jichan Chung, Dong Yin, Kannan Ramchandran

Keywords Paper

0

0

0

0

3:21

26/04/2020

Federated Adversarial Domain Adaptation

Xingchao Peng, Zijun Huang, Yizhe Zhu, Kate Saenko

Keywords Paper

Federated Learning, Domain Adaptation, Transfer Learning, Feature Disentanglement

0

0

0

2

4:57

22/09/2020

A federated recommender system for online services

Ben Tan, Bo Liu, Vincent Zheng, Qiang Yang

Keywords Paper

Recommender Systems, Federated Learning

0

0

0

0

1:40

26/04/2020

Federated Learning with Matched Averaging

Hongyi Wang, Mikhail Yurochkin, Yuekai Sun and
Dimitris Papailiopoulos, Yasaman Khazaeni

Keywords Paper

federated learning

0

0

0

0

13:54

06/12/2020

Personalized Federated Learning with Moreau Envelopes

Canh T. Dinh, Nguyen H. Tran, Josh Nguyen

Keywords Paper

0

0

0

0

3:17

06/12/2021

QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning

Kaan Ozkara, Navjot Singh, Deepesh Data, Suhas Diggavi

Keywords Paper

optimization, federated learning

0

0

0

1

14:51

06/12/2020

Ensemble Distillation for Robust Model Fusion in Federated Learning

Tao Lin, Lingjing Kong, Sebastian Stich, Martin Jaggi

Keywords Paper

0

0

0

0

2:59

02/02/2021

Dynamic Automaton-Guided Reward Shaping for Monte Carlo Tree Search

Alvaro Velasquez, Brett Bissey, Lior Barak and
Andre Beckus, Ismail Alkhouri, Daniel Melcer, George Atia

Keywords Paper

0

0

0

0

18:52

18/07/2021

Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning

Yann Fraboni, Richard Vidal, Laetitia Kameni, Marco Lorenzi

Keywords Paper

Optimization, Distributed and Parallel Optimization

0

0

0

0

5:01

14/09/2020

Social Influence Attentive Neural Network for Friend-Enhanced Recommendation

Yuanfu Lu, Ruobing Xie, Chuan Shi and
Yuan Fang, Wei Wang, Xu Zhang, Leyu Lin

Keywords Paper

heterogeneous graph, friend-enhanced recommendation, social influence

0

0

0

0

14:23

13/04/2021

Federated f-differential privacy

Qinqing Zheng, Shuxiao Chen, Qi Long, Weijie Su

Keywords Paper

0

0

0

0

3:08

18/07/2021

Ditto: Fair and Robust Federated Learning Through Personalization

Tian Li, Shengyuan Hu, Ahmad Beirami, Virginia Smith

Keywords Paper

, Algorithms, Large Scale Learning, Deep Learning, Embedding Approaches

0

0

0

0

5:27

13/04/2021

Convergence and accuracy trade-offs in federated learning and meta-learning

Zachary Charles, Jakub Konečný

Keywords Paper

0

0

0

0

3:04

22/11/2021

Model Composition: Can Multiple Neural Networks Be Combined into a Single Network Using Only Unlabeled Data?

Amin Banitalebi-Dehkordi, Xinyu Kang, Yong Zhang

Keywords Paper

Model Composition, Combining Neural Networks, Pseudo Label, Self Training, Label Aggregation, Combining Models

0

0

0

0

2:58

06/12/2020

AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning

Ximeng Sun, Rameswar Panda, Rogerio Feris, Kate Saenko

Keywords Paper

0

0

0

0

3:13

06/12/2021

Renyi Differential Privacy of The Subsampled Shuffle Model In Distributed Learning

Antonious Girgis, Deepesh Data, Suhas Diggavi

Keywords Paper

optimization, privacy, federated learning

0

0

0

1

15:50

14/06/2020

Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data

Xi Yan, David Acuna, Sanja Fidler

Keywords Paper

transfer learning, datasets, large-scale, domain, pre-training, search, self-supervised, classification, detection, segmentation

0

0

0

0

5:01

03/05/2021

FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning

Hong-You Chen, Wei-Lun Chao

Keywords Paper

0

0

0

0

5:06

06/12/2021

Breaking the centralized barrier for cross-device federated learning

Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale and
Mehryar Mohri, Sashank Reddi, Sebastian Stich, Ananda Theertha Suresh

Keywords Paper

optimization, reinforcement learning and planning, federated learning

0

0

0

0

13:48

06/12/2020

Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning

Filippos Christianos, Lukas Schäfer, Stefano Albrecht

Keywords Paper

, Applications -> Natural Language Processing

0

0

0

0

3:13

18/07/2021

Bayesian Structural Adaptation for Continual Learning

Abhishek Kumar, Sunabha Chatterjee, Piyush Rai

Keywords Paper

Probabilistic Methods, Bayesian Methods

0

0

0

0

7:39

03/05/2021

Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity

Jang-Hyun Kim, Wonho Choo, Hosan Jeong, Hyun Oh Song

Keywords Paper

Supervised Learning, Discrete Optimization, Data Augmentation, Deep Learning

0

0

0

0

14:43

02/02/2021

Lipschitz Lifelong Reinforcement Learning

Erwan Lecarpentier, David Abel, Kavosh Asadi and
Yuu Jinnai, Emmanuel Rachelson, Michael L. Littman

Keywords Paper

1

1

0

0

15:53

02/02/2021

A General Class of Transfer Learning Regression without Implementation Cost

Shunya Minami, Song Liu, Stephen Wu and
Kenji Fukumizu, Ryo Yoshida

Keywords Paper

0

0

0

0

14:13

03/05/2021

Federated Learning Based on Dynamic Regularization

Durmus Alp Emre Acar, Yue Zhao, Ramon Matas and
Matthew Mattina, Paul Whatmough, Venkatesh Saligrama

Keywords Paper

Distributed Optimization, Deep Neural Networks, Federated Learning

1

0

0

0

17:21

26/04/2020

Decentralized Deep Learning with Arbitrary Communication Compression

Anastasia Koloskova, Tao Lin, Sebastian U Stich, Martin Jaggi

Keywords Paper

0

0

0

0

5:16