skip to main content
10.1145/3539597.3570429acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

Position-Aware Subgraph Neural Networks with Data-Efficient Learning

Published: 27 February 2023 Publication History

Abstract

Data-efficient learning on graphs (GEL) is essential in real-world applications. Existing GEL methods focus on learning useful representations for nodes, edges, or entire graphs with "small" labeled data. But the problem of data-efficient learning for subgraph prediction has not been explored. The challenges of this problem lie in the following aspects: 1) It is crucial for subgraphs to learn positional features to acquire structural information in the base graph in which they exist. Although the existing subgraph neural network method is capable of learning disentangled position encodings, the overall computational complexity is very high. 2) Prevailing graph augmentation methods for GEL, including rule-based, sample-based, adaptive, and automated methods, are not suitable for augmenting subgraphs because a subgraph contains fewer nodes but richer information such as position, neighbor, and structure. Subgraph augmentation is more susceptible to undesirable perturbations. 3) Only a small number of nodes in the base graph are contained in subgraphs, which leads to a potential "bias" problem that the subgraph representation learning is dominated by these "hot" nodes. By contrast, the remaining nodes fail to be fully learned, which reduces the generalization ability of subgraph representation learning. In this paper, we aim to address the challenges above and propose a Position-Aware Data-Efficient Learning framework for subgraph neural networks called PADEL. Specifically, we propose a novel node position encoding method that is anchor-free, and design a new generative subgraph augmentation method based on a diffused variational subgraph autoencoder, and we propose exploratory and exploitable views for subgraph contrastive learning. Extensive experiment results on three real-world datasets show the superiority of our proposed method over state-of-the-art baselines.

Supplementary Material

MP4 File (20230131_211838.mp4)
Presentation video of 'Position-Aware Subgraph Neural Networks with Data Efficient Learning' (PADEL), WSDM '2023.

References

[1]
Emily Alsentzer, Samuel G. Finlayson, Michelle M. Li, and Marinka Zitnik. 2020. Subgraph Neural Networks. In NeurIPS.
[2]
Chris P. Austin and Hugh J. S. Dawkins. 2017. Medical research: Next decade's goals for rare diseases. NATURE, Vol. 548 (2017), 158--158.
[3]
Jean Bourgain. 1985. On Lipschitz embedding of finite metric spaces in Hilbert space. ISR J MATH, Vol. 52, 1--2 (1985), 46--52.
[4]
Conor A Bradley. 2020. A statistical framework for rare disease diagnosis. NAT REV GENET, Vol. 21, 1 (2020), 2--3.
[5]
Yaomin Chang, Chuan Chen, Weibo Hu, Zibin Zheng, Xiaocong Zhou, and Shouzhi Chen. 2022. Megnn: Meta-path extracted graph neural network for heterogeneous graph representation learning. KNOWL BASED SYST, Vol. 235 (2022), 107611.
[6]
Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020. Bias and Debias in Recommender System: A Survey and Future Directions. arXiv preprint, Vol. arXiv:2010.03240 (2020).
[7]
Edsger W. Dijkstra. 1959. A note on two problems in connexion with graphs. NUMER MATH, Vol. 1 (1959), 269--271.
[8]
Adam Paszke et al. 2019a. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In NeurIPS. 8024--8035.
[9]
Sebastian Köhler et al. 2019b. Expansion of the Human Phenotype Ontology (HPO) knowledge base and resources. NUCLEIC ACIDS RES, Vol. 47 (2019), D1018 -- D1027.
[10]
Michelle Girvan and Mark EJ Newman. 2002. Community structure in social and biological networks. P NATL ACAD SCI, Vol. 99, 12 (2002), 7821--7826.
[11]
Alex Graves, Santiago Ferná ndez, and Jü rgen Schmidhuber. 2005. Bidirectional LS™ Networks for Improved Phoneme Classification and Recognition. In ICANN (2), Vol. 3697. 799--804.
[12]
Aric A. Hagberg, Daniel A. Schult, and Pieter J. Swart. 2008. Exploring Network Structure, Dynamics, and Function using NetworkX. In SciPy2009. Pasadena, CA USA, 11 -- 15.
[13]
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation Learning on Graphs: Methods and Applications. IEEE DATA ENG BULL, Vol. 40, 3 (2017), 52--74.
[14]
Taila Hartley, Gabrielle Lemire, Kristin D. Kernohan, Heather E Howley, David R. Adams, and Kym M. Boycott. 2020. New Diagnostic Approaches for Undiagnosed Rare Genetic Diseases. ANNU REV GENOM HUM G (2020).
[15]
Kaveh Hassani and Amir Hosein Khas Ahmadi. 2020. Contrastive Multi-View Representation Learning on Graphs. In ICML, Vol. 119. 4116--4126.
[16]
Irina Higgins, Lo"i c Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In ICLR (Poster).
[17]
Dasol Hwang, Jinyoung Park, Sunyoung Kwon, Kyung-Min Kim, Jung-Woo Ha, and Hyunwoo J. Kim. 2020. Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs. In NeurIPS.
[18]
Zhiyi Jiang, Jianliang Gao, and Xinqi Lv. 2021. MetaP: Meta Pattern Learning for One-Shot Knowledge Graph Completion. In SIGIR. 2232--2236.
[19]
Unmesh Joshi and Jacopo Urbani. 2020. Searching for Embeddings in a Haystack: Link Prediction on Knowledge Graphs with Subgraph Pruning. In WWW. 2817--2823.
[20]
Pan-Jun Kim and Nathan D. Price. 2011. Genetic Co-Occurrence Network across Sequenced Microbes. PLOS COMPUT BIOL, Vol. 7 (2011).
[21]
Thomas N. Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. arXiv preprint, Vol. arXiv:1611.07308 (2016).
[22]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR (Poster).
[23]
Darong Lai, Zheyi Liu, Junyao Huang, Zhihong Chong, Weiwei Wu, and Christine Nardini. 2021. Attention Based Subgraph Classification for Link Prediction by Network Re-weighting. In CIKM. 3171--3175.
[24]
Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec. 2020. Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning. In NeurIPS.
[25]
Zheyi Liu, Darong Lai, Chuanyou Li, and Meng Wang. 2020. Feature Fusion Based Subgraph Classification for Link Prediction. In CIKM. 985--994.
[26]
Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In ICLR (Poster).
[27]
Sagar Maheshwari Marinka Zitnik, Rok Sosivc and Jure Leskovec. 2018. BioSNAP Datasets: Stanford Biomedical Network Dataset Collection. http://snap.stanford.edu/biodata.
[28]
Changping Meng, S. Chandra Mouli, Bruno Ribeiro, and Jennifer Neville. 2018. Subgraph Pattern Neural Networks for High-Order Graph Evolution Prediction. In AAAI. 3778--3787.
[29]
Diego P. P. Mesquita, Amauri H. Souza Jr., and Samuel Kaski. 2020. Rethinking pooling in graph neural networks. In NeurIPS.
[30]
Jianmo Ni, Larry Muhlstein, and Julian J. McAuley. 2019. Modeling Heart Rate and Activity Data for Personalized Fitness Recommendation. In WWW. 1343--1353.
[31]
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training. In KDD. 1150--1160.
[32]
Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. 2020. Self-Supervised Graph Transformer on Large-Scale Molecular Data. In NeurIPS.
[33]
Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. 2020. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In ICLR.
[34]
Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Yuanxing Ning, Philip S. Yu, and Lifang He. 2021. SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism. In WWW. ACM / IW3C2, 2081--2091.
[35]
Komal K. Teru, Etienne Denis, and Will Hamilton. 2020. Inductive Relation Prediction by Subgraph Reasoning. In ICML, Vol. 119. 9448--9457.
[36]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. JMLR, Vol. 9, 11 (2008).
[37]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS. 5998--6008.
[38]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR (Poster).
[39]
Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. In ICLR (Poster).
[40]
Ping Wang, Khushbu Agarwal, Colby Ham, Sutanay Choudhury, and Chandan K. Reddy. 2021a. Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks. In WWW. 2946--2957.
[41]
Xiao Wang, Nian Liu, Hui Han, and Chuan Shi. 2021b. Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning. In KDD. 1726--1736.
[42]
Xiyuan Wang and Muhan Zhang. 2022. GLASS: GNN with Labeling Tricks for Subgraph Representation Learning. In International Conference on Learning Representations. https://openreview.net/forum?id=XLxhEjKNbXj
[43]
Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. CHEMOMETR INTELL LAB, Vol. 2, 1--3 (1987), 37--52.
[44]
Tete Xiao, Xiaolong Wang, Alexei A. Efros, and Trevor Darrell. 2021. What Should Not Be Contrastive in Contrastive Learning. In ICLR.
[45]
Yaochen Xie, Zhao Xu, Zhengyang Wang, and Shuiwang Ji. 2021. Self-Supervised Learning of Graph Neural Networks: A Unified Review. arXiv preprint, Vol. arXiv:2102.10757 (2021).
[46]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In ICLR.
[47]
Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, and Jian Tang. 2021. Self-supervised Graph-level Representation Learning with Local and Global Structure. In ICML, Vol. 139. 11548--11558.
[48]
Jiaxuan You, Rex Ying, and Jure Leskovec. 2019. Position-aware Graph Neural Networks. In ICML, Vol. 97. 7134--7143.
[49]
Yuning You, Tianlong Chen, Yang Shen, and Zhangyang Wang. 2021. Graph Contrastive Learning Automated. In ICML, Vol. 139. 12121--12132.
[50]
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In NeurIPS.
[51]
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. 2021. On Explainability of Graph Neural Networks via Subgraph Explorations. In ICML, Vol. 139. 12241--12252.
[52]
Jiaqi Zeng and Pengtao Xie. 2021. Contrastive Self-supervised Learning for Graph Classification. In AAAI. 10824--10832.
[53]
Chuxu Zhang, Jundong Li, and Meng Jiang. 2021. Data Efficient Learning on Graphs. In KDD. 4092--4093.
[54]
Tong Zhao, Yozen Liu, Leonardo Neves, Oliver J. Woodford, Meng Jiang, and Neil Shah. 2021. Data Augmentation for Graph Neural Networks. In AAAI. 11015--11023.
[55]
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI Open, Vol. 1 (2020), 57--81.
[56]
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph Contrastive Learning with Adaptive Augmentation. In WWW. 2069--2080.

Cited By

View all

Index Terms

  1. Position-Aware Subgraph Neural Networks with Data-Efficient Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining
    February 2023
    1345 pages
    ISBN:9781450394079
    DOI:10.1145/3539597
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 February 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. contrastive learning
    2. data-efficient learning
    3. generative model
    4. subgraph neural networks

    Qualifiers

    • Research-article

    Funding Sources

    • Shanghai Municipal Science and Technology Major Project, China
    • Natural Science Foundation of China
    • Huawei Technologies

    Conference

    WSDM '23

    Acceptance Rates

    Overall Acceptance Rate 498 of 2,863 submissions, 17%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)107
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 14 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media