23/08/2020

GPT-GNN: Generative pre-training of graph neural networks

Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, Yizhou Sun

Keywords: network embedding, graph representation learning, gnn pre-training, graph neural networks, generative pre-training

Abstract Paper Similar Papers

Abstract: Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data. However, training GNNs requires abundant task-specific labeled data, which is often arduously expensive to obtain. One effective way to reduce the labeling effort is to pre-train an expressive GNN model on unlabelled data with self-supervision and then transfer the learned model to downstream tasks with only a few labels. In this paper, we present the GPT-GNN framework to initialize GNNs by generative pre-training. GPT-GNN introduces a self-supervised attributed graph generation task to pre-train a GNN so that it can capture the structural and semantic properties of the graph. We factorize the likelihood of graph generation into two components: 1) attribute generation and 2) edge generation. By modeling both components, GPT-GNN captures the inherent dependency between node attributes and graph structure during the generative process. Comprehensive experiments on the billion-scale open academic graph and Amazon recommendation data demonstrate that GPT-GNN significantly outperforms state-of-the-art GNN models without pre-training by up to 9.1

The video of this talk cannot be embedded. You can watch it here:
https://dl.acm.org/doi/10.1145/3394486.3403237#sec-supp
(Link will open in new window)
 0
 0
 0
 0
  Share    
This is an embedded video. Talk and the respective paper are published at KDD 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd Characters remaining: 140

Similar Papers

 0
 0
 0
 0
 14:49 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 9:39 
 0
 0
 0
 0
 8:23 
 0
 0
 0
 0
 15:10 
 0
 0
 0
 0
 19:59 
 0
 0
 0
 0
 7:06 
 0
 0
 0
 0
 1:00 
 0
 0
 0
 0
 14:36 
 0
 0
 0
 0
 14:16 
 0
 0
 0
 0
 16:50 
 0
 0
 0
 0
 9:36 
 0
 0
 0
 0
 8:20 
 0
 0
 0
 0
 3:24 
 0
 0
 0
 0
 13:59 
 0
 0
 0
 0
 3:17 
 0
 0
 0
 0
 9:20 
 0
 0
 0
 0
 1:03 
 0
 0
 0
 0
 2:50 
 0
 0
 0
 0
 11:16 
 0
 0
 0
 0
 8:47 
 0
 0
 0
 0
 7:56 
 0
 0
 0
 0
 13:18 
 0
 0
 0
 0
 4:39 
 0
 0
 0
 0
 5:07 
 0
 0
 0
 0
 3:20 
 0
 0
 0
 0
 14:31 
 0
 0
 0
 0
 14:56 
 0
 0
 0
 0
 14:18 
 0
 0
 0
 0
 12:14 
 0
 0
 0
 0
 3:15 
 0
 0
 0
 0
 3:18 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 4:53 
 0
 0
 0
 0
 18:16 
 0
 0
 0
 0
 3:23 
 0
 0
 0
 0
 16:41 
 0
 0
 0
 0
 18:52 
 0
 0
 0
 0
 5:54 
 0
 0
 0
 0
 13:35