research-article

Public Access

XGNN: Towards Model-Level Explanations of Graph Neural Networks

Authors:

Shuiwang JiAuthors Info & Claims

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 430 - 438

https://doi.org/10.1145/3394486.3403085

Published: 20 August 2020 Publication History

Abstract

Graphs neural networks (GNNs) learn node features by aggregating and combining neighbor information, which have achieved promising performance on many graph tasks. However, GNNs are mostly treated as black-boxes and lack human intelligible explanations. Thus, they cannot be fully trusted and used in certain application domains if GNN models cannot be explained. In this work, we propose a novel approach, known as XGNN, to interpret GNNs at the model-level. Our approach can provide high-level insights and generic understanding of how GNNs work. In particular, we propose to explain GNNs by training a graph generator so that the generated graph patterns maximize a certain prediction of the model. We formulate the graph generation as a reinforcement learning task, where for each step, the graph generator predicts how to add an edge into the current graph. The graph generator is trained via a policy gradient method based on information from the trained GNNs. In addition, we incorporate several graph rules to encourage the generated graphs to be valid. Experimental results on both synthetic and real-world datasets show that our proposed methods help understand and verify the trained GNNs. Furthermore, our experimental results indicate that the generated graphs can provide guidance on how to improve the trained GNNs.

Supplementary Material

MP4 File (3394486.3403085.mp4)

Presentation Video.

Download
198.48 MB

References

[1]

Uri Alon. 2006. An introduction to systems biology: design principles of biological circuits .Chapman and Hall/CRC.

[2]

Uri Alon. 2007. Network motifs: theory and experimental approaches. Nature Reviews Genetics, Vol. 8, 6 (2007), 450.

[3]

Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, Vol. 10, 7 (2015).

[4]

Federico Baldassarre and Hossein Azizpour. 2019. Explainability Techniques for Graph Convolutional Networks. In International Conference on Machine Learning (ICML) Workshops, 2019 Workshop on Learning and Reasoning with Graph-Structured Representations.

[5]

Piotr Dabkowski and Yarin Gal. 2017. Real time image saliency for black box classifiers. In Advances in Neural Information Processing Systems. 6967--6976.

[6]

Asim Kumar Debnath, Rosa L Lopez de Compadre, Gargi Debnath, Alan J Shusterman, and Corwin Hansch. 1991. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, Vol. 34, 2 (1991), 786--797.

[7]

Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).

[8]

Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. Technical Report, University of Montreal, Vol. 1341, 3 (2009), 1.

[9]

Ruth C Fong and Andrea Vedaldi. 2017. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE International Conference on Computer Vision. 3429--3437.

[10]

Hongyang Gao and Shuiwang Ji. 2019 a. Graph representation learning via hard and channel-wise attention networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 741--749.

Digital Library

[11]

Hongyang Gao and Shuiwang Ji. 2019 b. Graph U-Net. In International conference on machine learning. 2083--2092.

[12]

Muriel Gevrey, Ioannis Dimopoulos, and Sovan Lek. 2003. Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecological modelling, Vol. 160, 3 (2003), 249--264.

[13]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 1263--1272.

Digital Library

[14]

Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik. 2017. Objective-reinforced generative adversarial networks (ORGAN) for sequence generation models. arXiv preprint arXiv:1705.10843 (2017).

[15]

Aric Hagberg, Pieter Swart, and Daniel S Chult. 2008. Exploring network structure, dynamics, and function using NetworkX. Technical Report. Los Alamos National Lab.(LANL), Los Alamos, NM (United States).

[16]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in neural information processing systems. 1024--1034.

[17]

Wengong Jin, Regina Barzilay, and Tommi Jaakkola. 2018. Junction Tree Variational Autoencoder for Molecular Graph Generation. In Proceedings of the 35th International Conference on Machine Learning. 2323--2332.

[18]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations.

[19]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the International Conference on Learning Representations.

[20]

Junhyun Lee, Inyeop Lee, and Jaewoo Kang. 2019. Self-Attention Graph Pooling. In International Conference on Machine Learning. 3734--3743.

[21]

Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing Neural Predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 107--117.

[22]

Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. 2018. Learning deep generative models of graphs. arXiv preprint arXiv:1803.03324 (2018).

[23]

Ron Milo, Shai Shen-Orr, Shalev Itzkovitz, Nadav Kashtan, Dmitri Chklovskii, and Uri Alon. 2002. Network motifs: simple building blocks of complex networks. Science, Vol. 298, 5594 (2002), 824--827.

[24]

Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski. 2017. Plug & play generative networks: Conditional iterative generation of images in latent space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4467--4477.

[25]

Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 427--436.

[26]

Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. 2017. Feature visualization. Distill, Vol. 2, 11 (2017), e7.

[27]

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. In Proceedings of the International Conference on Learning Representations.

[28]

Spencer L Seager and Michael R Slabaugh. 2013. Chemistry for today: General, organic, and biochemistry. Cengage learning.

[29]

Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision. 618--626.

[30]

Shai S Shen-Orr, Ron Milo, Shmoolik Mangan, and Uri Alon. 2002. Network motifs in the transcriptional regulation network of Escherichia coli. Nature genetics, Vol. 31, 1 (2002), 64.

[31]

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013).

[32]

Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017).

[33]

Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. 2014. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806 (2014).

[34]

James Andrew Storer. 2012. An introduction to data structures and algorithms .Springer Science & Business Media.

[35]

Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems. 1057--1063.

[36]

Kiran K Thekumparampil, Chong Wang, Sewoong Oh, and Li-Jia Li. 2018. Attention-based graph neural network for semi-supervised learning. arXiv preprint arXiv:1803.03735 (2018).

[37]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations. https://openreview.net/forum?id=rJXMpikCZ

[38]

Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. GraphGAN: Graph representation learning with generative adversarial nets. In Thirty-Second AAAI Conference on Artificial Intelligence. 2508--2515.

[39]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In International Conference on Learning Representations. https://openreview.net/forum?id=ryGs6iA5Km

[40]

Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. 2019. GNNExplainer: Generating Explanations for Graph Neural Networks. In Advances in Neural Information Processing Systems 32. 9244--9255.

[41]

Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. 2018. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems. 6410--6421.

[42]

L Yu, W Zhang, J Wang, and Y Yu. 2017. Seqgan: sequence generative adversarial nets with policy gradient. In AAAI-17: Thirty-First AAAI Conference on Artificial Intelligence, Vol. 31. Association for the Advancement of Artificial Intelligence (AAAI), 2852--2858.

[43]

Hao Yuan, Yongjun Chen, Xia Hu, and Shuiwang Ji. 2019. Interpreting Deep Models for Text Analysis via Optimization and Regularization Methods. In Thirty-Third AAAI Conference on Artificial Intelligence. 5717--5724.

Digital Library

[44]

Hao Yuan and Shuiwang Ji. 2020. StructPool: Structured Graph Pooling via Conditional Random Fields. In International Conference on Learning Representations. https://openreview.net/forum?id=BJxg_hVtwH

[45]

Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In European Conference on Computer Vision. Springer, 818--833.

[46]

Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. In Advances in Neural Information Processing Systems. 5165--5175.

[47]

Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An end-to-end deep learning architecture for graph classification. In Thirty-Second AAAI Conference on Artificial Intelligence. 4438--4445.

[48]

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE International Conference on Computer Vision. 2921--2929.

Cited By

Liu XHe YTai WXu XZhou FLuo G(2025)Exploring the Chameleon Effect of Contextual Dynamics in Temporal Knowledge Graph for Event PredictionTsinghua Science and Technology10.26599/TST.2024.901006730:1(433-455)Online publication date: Feb-2025
https://doi.org/10.26599/TST.2024.9010067
Longa AAzzolin SSantin GCencetti GLio PLepri BPasserini A(2025)Explaining the Explainers in Graph Neural Networks: a Comparative StudyACM Computing Surveys10.1145/369644457:5(1-37)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3696444
Lu HWang LMa XCheng JZhou M(2025)A survey of graph neural networks and their industrial applicationsNeurocomputing10.1016/j.neucom.2024.128761614(128761)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128761
Show More Cited By

Index Terms

XGNN: Towards Model-Level Explanations of Graph Neural Networks
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Mathematics of computing
  1. Discrete mathematics
    1. Graph theory
      1. Graph algorithms

Recommendations

Toward the analysis of graph neural networks
ICSE-NIER '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results

Graph Neural Networks (GNNs) have recently emerged as an effective framework for representing and analyzing graph-structured data. GNNs have been applied to many real-world problems such as knowledge graph analysis, social networks recommendation, and ...
Graph Neural Networks: Foundation, Frontiers and Applications
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

The field of graph neural networks (GNNs) has seen rapid and incredible strides over the recent years. Graph neural networks, also known as deep learning on graphs, graph representation learning, or geometric deep learning, have become one of the fastest-...
View-based Explanations for Graph Neural Networks
SIGMOD

Generating explanations for graph neural networks (GNNs) has been studied to understand their behaviors in analytical tasks such as graph classification. Existing approaches aim to understand the overall results of GNNs rather than providing explanations ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

August 2020

3664 pages

ISBN:9781450379984

DOI:10.1145/3394486

General Chairs:
Rajesh Gupta
UC San Diego, USA
,
Yan Liu
USC, USA
,
Program Chairs:
Mohak Shah
LG Electronics, USA
,
Suju Rajan
Linkedin, USA
,
Publications Chairs:
Jiliang Tang
Michigan State, USA
,
B. Aditya Prakash
Georgia Tech, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation
Defense Advanced Research Projects Agency

Conference

KDD '20

Sponsor:

KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

July 6 - 10, 2020

CA, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

184
Total Citations
View Citations
5,580
Total Downloads

Downloads (Last 12 months)1,553
Downloads (Last 6 weeks)155

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu XHe YTai WXu XZhou FLuo G(2025)Exploring the Chameleon Effect of Contextual Dynamics in Temporal Knowledge Graph for Event PredictionTsinghua Science and Technology10.26599/TST.2024.901006730:1(433-455)Online publication date: Feb-2025
https://doi.org/10.26599/TST.2024.9010067
Longa AAzzolin SSantin GCencetti GLio PLepri BPasserini A(2025)Explaining the Explainers in Graph Neural Networks: a Comparative StudyACM Computing Surveys10.1145/369644457:5(1-37)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3696444
Lu HWang LMa XCheng JZhou M(2025)A survey of graph neural networks and their industrial applicationsNeurocomputing10.1016/j.neucom.2024.128761614(128761)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128761
Siachos IKanakaris NKaracapilidis N(2025)Software bug prediction using graph neural networks and graph-based text representationsExpert Systems with Applications10.1016/j.eswa.2024.125290259(125290)Online publication date: Jan-2025
https://doi.org/10.1016/j.eswa.2024.125290
Ma YLiu XGuo CJin BLiu H(2025)KnowGNN: a knowledge-aware and structure-sensitive model-level explainer for graph neural networksApplied Intelligence10.1007/s10489-024-06034-455:2Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s10489-024-06034-4
Kamal ARobardet CPlantevit M(2025)Game Theoretic Explanations for Graph Neural NetworksMachine Learning and Principles and Practice of Knowledge Discovery in Databases10.1007/978-3-031-74633-8_14(217-232)Online publication date: 1-Jan-2025
https://doi.org/10.1007/978-3-031-74633-8_14
Li JPang MDong YJia JWang BSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Graph neural network explanations are fragileProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693216(28551-28567)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693216
Chen YBian YHan BCheng JSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)How interpretable are interpretable graph neural networks?Proceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692318(6413-6456)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692318
Bui NNguyen HNguyen VYing RSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Explaining graph neural networks via structure-aware interaction indexProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692263(4854-4883)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692263
Karamanou ABrimos PKalampokis ETarabanis K(2024)Explainable Graph Neural Networks: An Application to Open Statistics Knowledge Graphs for Estimating House PricesTechnologies10.3390/technologies1208012812:8(128)Online publication date: 6-Aug-2024
https://doi.org/10.3390/technologies12080128
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents