Graph Quaternion-Valued Attention Networks for Node Classification
Pages 673 - 677
Abstract
Node classification is a prominent graph-based task and various Graph neural networks (GNNs) models have been applied for solving it. In this paper, we introduce a novel GNN architecture for node classification called Graph Quaternion-Valued Attention Networks (GQAT), which enhances the original graph attention networks by replacing the vector multiplication in self-attention with quaternion vector multiplication.
One of the primary advantages of GQAT is the significant reduction in model parameters, as quaternion operations require only 1/4 of the calculation matrix, contributing to a more lightweight model. Moreover, GQAT excels at capturing intricate relationships between nodes, owing to the sophisticated nature of quaternion operations. We conduct extensive experiments on Cora, Citeseer, and Pubmed for node classification. The results demonstrate that GQAT outperforms conventional graph attention networks in terms of node classification accuracy while requiring fewer parameters.
References
[1]
Dominik-Alexander Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. Fast and accurate deep network learning by exponential linear units (ELUs). arxiv:1511.07289 [cs.LG]
[2]
PSL DataShop. [n. d.]. KDD Cup 2003 dataset. Retrieved August 5, 2021 from https://pslcdatashop.web.cmu.edu/KDDCup/downloads.jsp
[3]
Micha"el Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, Vol. 29. https://doi.org/10.5555/3157382.3157527
[4]
Shobeir Fakhraei, Bowen Huang, Louiqa Raschid, and Lise Getoor. 2014. Network-based drug-target interaction prediction with probabilistic soft logic. IEEE/ACM Trans. Comput. Biol. Bioinformatics 11, 5 (Sep-Oct. 2014), 775–787. https://doi.org/10.1109/TCBB.2014.2325031
[5]
Chase J Gaudet and Anthony S Maida. 2018. Deep quaternion networks. In Advances in Neural Information Processing Systems, Vol. 29. 1–8. https://doi.org/10.5555/3157382.3157527
[6]
Eleonora Grassucci, Edoardo Cicero, and Danilo Comminiello. 2022. Quaternion generative adversarial networks. In Generative Adversarial Learning: Architectures and Applications. 57–86. https://doi.org/10.1007/978-3-030-91390-8_4
[7]
W. R. Hamilton. 1944. Quaternions. Proc. Royal Irish Acad. 50, 1 (1944), 89–92.
[8]
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arxiv:1609.02907 [cs.LG]
[9]
Titouan Parcollet, Mirco Ravanelli, Mohamed Morchid, Georges Linar‘es, Chiheb Trabelsi, Renato De Mori, and Yoshua Bengio. 2018. Quaternion recurrent neural networks. arxiv:1806.04418 [cs.LG]
[10]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701–710. https://doi.org/10.1145/2623330.2623732
[11]
Daniel Pletinckx. 1989. Quaternion calculus as a basic tool in computer graphics. Vis. Comput. 5, 1-2 (1989), 2–13. https://doi.org/10.1007/BF01901476
[12]
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors. Nature 323, 6088 (1986), 533–536. https://doi.org/10.1038/323533a0
[13]
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI Magazine 29, 3 (2008), 93–93. https://doi.org/10.1609/aimag.v29i3.2157
[14]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017). https://doi.org/10.5555/3295222.3295349
[15]
Petar Veličkovi’c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arxiv:1710.10903 [cs.LG]
[16]
Bo Xu, Lianzhao Wang, Shengxin Li, and Jiao Zhang. 2020. A novel calibration method of SINS/DVL integration navigation system based on quaternion. IEEE Sensors J. 20, 16 (2020), 9567–9580. https://doi.org/10.1109/JSEN.2020.2988500
[17]
Jin Xu, Zishan Li, Bowen Du, Miaomiao Zhang, and Jing Liu. 2020. Reluplex made more practical: Leaky ReLU. In 2020 IEEE Symposium on Computers and communications (ISCC). 1–7. https://doi.org/10.1109/ISCC50000.2020.9219587
[18]
Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2016. Revisiting semi-supervised learning with graph embeddings. In Proceedings of the International Conference on Machine Learning. 40–48. https://doi.org/10.5555/3045390.3045396
[19]
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI open 1 (2020), 57–81. https://doi.org/10.1016/j.aiopen.2021.01.001
[20]
Xuanyu Zhu, Yi Xu, Hongteng Xu, and Changjian Chen. 2018. Quaternion convolutional neural networks. In Proceedings of the European Conference on Computer Vision (ECCV). 631–647.
Index Terms
- Graph Quaternion-Valued Attention Networks for Node Classification
Recommendations
Label-Consistency based Graph Neural Networks for Semi-supervised Node Classification
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information RetrievalGraph neural networks (GNNs) achieve remarkable success in graph-based semi-supervised node classification, leveraging the information from neighboring nodes to improve the representation learning of target node. The success of GNNs at node ...
GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data MiningNode classification is an important research topic in graph learning. Graph neural networks (GNNs) have achieved state-of-the-art performance of node classification. However, existing GNNs address the problem where node samples for different classes are ...
Node Classification of Graph Neural Networks Based on Graph Degree-Symmetry
AAIA '23: Proceedings of the 2023 International Conference on Advances in Artificial Intelligence and ApplicationsIn order to accurately reflect the information of each node's neighbours and the topological relationship of the graph structure, the graph neural network node classification task aims to treat each node and all of its neighbouring nodes as a subgraph ...
Comments
Information & Contributors
Information
Published In
May 2023
1025 pages
ISBN:9798400700705
DOI:10.1145/3603781
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 July 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CNIOT'23
CNIOT'23: 2023 4th International Conference on Computing, Networks and Internet of Things
May 26 - 28, 2023
Xiamen, China
Acceptance Rates
Overall Acceptance Rate 39 of 82 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 50Total Downloads
- Downloads (Last 12 months)31
- Downloads (Last 6 weeks)1
Reflects downloads up to 09 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format