research-article

Variational Graph Autoencoder with Adversarial Mutual Information Learning for Network Representation Learning

Authors:

Guang LianAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 17, Issue 3

Article No.: 45, Pages 1 - 18

https://doi.org/10.1145/3555809

Published: 20 March 2023 Publication History

Abstract

With the success of Graph Neural Network (GNN) in network data, some GNN-based representation learning methods for networks have emerged recently. Variational Graph Autoencoder (VGAE) is a basic GNN framework for network representation. Its purpose is to well preserve the topology and node attribute information of the network to learn node representation, but it only reconstructs network topology, and does not consider the reconstruction of node features. This strategy will make node representation can not well reserve node features information, impairing the ability of the VGAE method to learn higher quality representations. To solve this problem, we arise a new network representation method to improve the VGAE method for well retaining both node features and network structure information. The method utilizes adversarial mutual information learning to maximize the mutual information (MI) of node features and node representations during the encoding process of the variational autoencoder, which forces the variational encoder to get the representation containing the most informative node features. The method consists of three parts: a variational graph autoencoder includes a variational encoder (MI generator (G)) and a decoder, a positive MI sample module (maximizing MI module), and an MI discriminator (D). Furthermore, we explain why maximizing MI between node features and node representation can reconstruct node attributes. Finally, we conduct experiments on seven public representative datasets for nodes classification, nodes clustering, and graph visualization tasks. Experimental results demonstrate that the proposed algorithm significantly outperforms current popular network representation algorithms on these tasks. The best improvement is 17.13% than the VGAE method.

References

[1]

M. B. Hastings. 2006. Community detection as an inference problem. Physical Review E74, 3 (2006), 035–102.

[2]

T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the ICLR.

[3]

R. Ying, R. N. He, K. F. Chen, et al. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the SIGKDD Graph Convolutional Neural Networks for Web-scale Recommender Systems. 974–983.

Digital Library

[4]

S. S. Cao, W. Lu, and Q. K. Xu. 2015. GraRep: Learning graph representations with global structural information. In Proceedings of the GraRep: Learning Graph Representations with Global Structural Information. 891–900.

Digital Library

[5]

Y. Li, C. F. Sha, X. Huang, and Y. C. Zhang. 2018. Community detection in attributed graphs: An embedding approach. In Proceedings of the AAAIConference on Artificial Intelligence. 338–345.

[6]

C. Yang, Z. Y. Liu, D. L. Zhao, et al. 2015. Network representation learning with rich text information. In Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2111–2117.

Digital Library

[7]

J. Qiu, Y. Dong, H. Ma, et al. 2019. Netsmf: Large-scale network embedding as sparse matrix factorization. In Proceedings of the 28th International Conference on World Wide Web. San Francisco. 1509–1520.

Digital Library

[8]

J. Zhang, Y. Dong, Y. Wang, et al. 2019. ProNE: fast and scalable network representation learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao. 4278–4284.

[9]

B. Perozzi, R. AlRfou, and S. Skiena. 2014. Deep walk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701–710.

Digital Library

[10]

A. Grover and J. Leskovec. 2016. Node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.

Digital Library

[11]

J. Tang, M. Qu, M. Wang, et al. 2015. Line: Large scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. Florence, Italy. 1067–1077.

Digital Library

[12]

B. Perozzi, V. Kulkarni, H. Chen, and S. Skiena. 2017. Don’t walk, skip! Online learning of multi-scale network embeddings. In Proceedings of the 2017 IEEE/ACM International Conference.

[13]

B. Rozemberczki and R. Sarkar. 2018. Fast sequence-based embedding with diffusion graphs. In Proceedings of the International Workshop on Complex Networks. Springer (2018). 99–107.

[14]

C. Wang, S. Pan, G.g Long, et al. 2017. Mgae: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 889–898.

Digital Library

[15]

L. Ribeiro, P. Saverese, and D. Figueiredo. 2017. Struc2vec: Learning node representations from structural identity. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. 385–394.

Digital Library

[16]

D. Wang, P. Cui, and W. Zhu. 2016. Structural deep network embedding. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery \(and\) Data Mining. 1225–1234.

Digital Library

[17]

P. Velickovič, G. Cucurull, A. Casanova, A. Romero, P. Ĺio, and Y. Bengio. 2018. Graph attention networks. In Proceedings of the International Conference on Learning Representations.

[18]

W. Hamilton, Z. Ying, and J. Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the Advances in Neural Information Processing Systems 1024–1034.

[19]

T. N. Kipf and M. Welling. 2016. Variational graph auto-encoders. In Proceedings of the NIPS Workshop on Bayesian Deep Learning.

[20]

S. Pan, R. Hu, G. Long, et al. 2018. Adversarially regularized graph autoencoder for graph embedding. In Proceedings of the IJCAI. 2609–2615.

[21]

W. C. Yu, C. Zheng, W. Cheng, D. J. Song, et al. 2018. Learning deep network representations with adversarially regularized autoencoders. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. London, United Kingdom 2663–2671.

Digital Library

[22]

D. P. Kingma and M. Welling. 2014. Auto-Encoding Variational Bayes. arXiv:1312.6114v10. Retrieved from https://arxiv.org/abs/1312.6114v10

[23]

J. You, R. Ying, and J. Leskovec. 2019. Position-aware graph neural networks. In Proceedings of the International Conference on Machine Learning. 7134–7143.

[24]

K. M. Borgwardt and H. P. Kriegel. 2005. Shortest-path kernels on graphs. In Proceedings of the International Conference on Data Mining.

Digital Library

[25]

N. Shervashidze, S. Vishwanathan, T. Petri, et al. 2009. Efficient graphlet kernels for large graph comparison. In Proceedings of the Artificial Intelligence and Statistics. 488–495.

[26]

N. Shervashidze, P. Schweitzer, E. J. Leeuwen, et al. 2011. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research. 2539–2561.

[27]

P. Yanardag and S. Vishwana. 2015. Deep graph kernels. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. 1365–1374.

Digital Library

[28]

R. Kondor and H. Pan. 2016. The multiscale laplacian graph kernel. In Proceedings of the Advances in Neural Information Processing Systems. 2990–2998.

[29]

J. Park, M. Lee, H. J. Chang, et al. 2019. Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In Proceedings of the International Conference on Computer Vision. 6519–6528.

[30]

P. Veli’ckovic, W. Fedus, et al. 2019. Deep graph infomax. In Proceedings of the International Conference on Learning Representations.

[31]

F. Y. Sun, J. Hoffman, Verma, et al. 2020. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In Proceedings of the International Conference on Learning Representations.

[32]

R. D. Hjelm, A. Fedorov, Lavoie-Marchildon, et al. 2019. Learning deep representations by mutual information estimation and maximization. In Proceedings of the International Conference on Learning Representations.

[33]

I. Goodfellow, J. PougetAbadie, M. Mirza, et al. 2014. Generative adversarial nets. In Proceedings of the Advances in Neural Information Processing Systems. 2672–2680.

[34]

N. Sebastian, C. Botond and T. Ryota. 2016. F-GAN: Training generative neural samplers using variational divergence minimization. In Proceedings of the 30th International Conference on Neural Information Processing Systems. 271–279.

[35]

Retrieved from https://spaces.ac.cn/archives/6088.

[36]

M. Defferrard, X. Bresson, and P. Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Proceedings of the Advances in Neural Information Processing Systems. 384–852.

[37]

F. Monti, D. Boscaini, J. Masci, E. Rodola, et al. 2017. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the Conference on Computer Vision and Pattern Recognition.

[38]

X. Huang, J. Li and X. Hu. 2017. Accelerated attributed network embedding. In Proceedings of the 2017 SIAM International Conference on Data Mining. Houston, Texas. 633–641.

[39]

L. Liao, X. He, H. Zhang, et al. 2018. Attributed social network embedding. IEEE Transactions on Knowledge and Data Engineering. 2257–2270.

[40]

D. He, L. Zhai, Z. Li, et al. 2020. Adversarial mutual information learning for network embedding. In Proceedings of the 29th International Joint Conference on Artificial Intelligence.

[41]

X. Shu, G. J. Qi, J. Tang, and J. Wang. 2015. Weekly-shared deep transfer networks for heterogeneous-domain knowledge propagation. ACM International Conference on Multimedia. 35–44.

[42]

J. Tang, X. Shu, Z. Li, G. J. Qi, et al. 2016. Generalized deep transfer networks for heterogeneous-domain knowledge propagation. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 4s (2016), 68.

Digital Library

[43]

W. Huang, Y. Li, Y. Fang, J. Fan, et al. 2020. BiANE: Bipartite attributed network embedding. International ACM SIGIR Conference on Research and Development in Information Retrieval.

[44]

Z. Liu, C. Huang, Y. Yu, and J. Dong. 2021. Motif-preserving dynamic attributed network embedding. International World Wide Web Conferences.

[45]

T. He, L. Gao, J. Song, et al. 2020. SNEQ: Semi-supervised attributed network embedding with attention-based quantisation. In Proceedings of the AAAI Conference on Artificial Intelligence. 4091–4098.

Cited By

Fofanah ALeigh A(2025)EATSA-GNN: Edge-Aware and Two-Stage attention for enhancing graph neural networks based on teacher–student mechanisms for graph node classificationNeurocomputing10.1016/j.neucom.2024.128686612(128686)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128686
Zhang SMeng XZhang Y(2024)Variational Type Graph Autoencoder for Denoising on Event RecommendationACM Transactions on Information Systems10.1145/370315643:1(1-27)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3703156
Liu XHao YZhao LLiu GSheng VZhao P(2024)LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive LearningACM Transactions on Knowledge Discovery from Data10.1145/365730218:7(1-24)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1145/3657302
Show More Cited By

Index Terms

Variational Graph Autoencoder with Adversarial Mutual Information Learning for Network Representation Learning
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

Adversarial learning based residual variational graph normalized autoencoder for network representation
Abstract
With the success of Graph Neural Networks (GNNs) on non-Euclidean data, some GNN-based approaches to network representation learning have emerged in recent years. Graph autoencoder (GAE) and variational GAE (VGAE) are unsupervised learning ...
Adversarial correlated autoencoder for unsupervised multi-view representation learning
Abstract
To eliminate the view discrepancy of multi-view data due to different distributions, the key is to learn the common representation for multi-view data in many practical applications. To achieve the end, we propose a novel unsupervised ...
Text-enhanced network representation learning
Abstract
Network representation learning called NRL for short aims at embedding various networks into low-dimensional continuous distributed vector spaces. Most existing representation learning methods focus on learning representations purely based on the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 17, Issue 3

April 2023

379 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3583064

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 March 2023

Online AM: 22 August 2022

Accepted: 31 July 2022

Revised: 23 July 2022

Received: 29 January 2022

Published in TKDD Volume 17, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of Guangdong Province
Science and Technology Program of Guangzhou City

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
941
Total Downloads

Downloads (Last 12 months)287
Downloads (Last 6 weeks)18

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fofanah ALeigh A(2025)EATSA-GNN: Edge-Aware and Two-Stage attention for enhancing graph neural networks based on teacher–student mechanisms for graph node classificationNeurocomputing10.1016/j.neucom.2024.128686612(128686)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128686
Zhang SMeng XZhang Y(2024)Variational Type Graph Autoencoder for Denoising on Event RecommendationACM Transactions on Information Systems10.1145/370315643:1(1-27)Online publication date: 5-Nov-2024
https://dl.acm.org/doi/10.1145/3703156
Liu XHao YZhao LLiu GSheng VZhao P(2024)LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive LearningACM Transactions on Knowledge Discovery from Data10.1145/365730218:7(1-24)Online publication date: 19-Jun-2024
https://dl.acm.org/doi/10.1145/3657302
Fu CYu PYu YHuang CZhao ZDong J(2024)MHGCN+: Multiplex Heterogeneous Graph Convolutional NetworkACM Transactions on Intelligent Systems and Technology10.1145/365004615:3(1-25)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3650046
Yi JChen Z(2024)Deconfounded Cross-modal Matching for Content-based Micro-video Background Music RecommendationACM Transactions on Intelligent Systems and Technology10.1145/365004215:3(1-25)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3650042
Zheng YJia C(2024)ProtoMGAE: Prototype-Aware Masked Graph Auto-Encoder for Graph Representation LearningACM Transactions on Knowledge Discovery from Data10.1145/364914318:6(1-22)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3649143
Gao CZheng YWang WFeng FHe XLi Y(2024)Causal Inference in Recommender Systems: A Survey and Future DirectionsACM Transactions on Information Systems10.1145/363904842:4(1-32)Online publication date: 9-Feb-2024
https://dl.acm.org/doi/10.1145/3639048
Han DHuang YLiu JLiao KLin K(2024)LSAB: User Behavioral Pattern Modeling in Sequential Recommendation by Learning Self-Attention BiasACM Transactions on Knowledge Discovery from Data10.1145/363262518:3(1-20)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3632625
Neloy ATurgeon M(2024)A comprehensive study of auto-encoders for anomaly detection: Efficiency and trade-offsMachine Learning with Applications10.1016/j.mlwa.2024.10057217(100572)Online publication date: Sep-2024
https://doi.org/10.1016/j.mlwa.2024.100572
Zhao JHe DGe MHuang YShan LQin YFeng Z(2024)GA-GGD: Improving semantic discriminability in graph contrastive learning via Generative Adversarial NetworkInformation Fusion10.1016/j.inffus.2024.102465110(102465)Online publication date: Oct-2024
https://doi.org/10.1016/j.inffus.2024.102465
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents