research-article

Profiling the Design Space for Graph Neural Networks based Collaborative Filtering

Authors:

Chuan ShiAuthors Info & Claims

WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Pages 1109 - 1119

https://doi.org/10.1145/3488560.3498520

Published: 15 February 2022 Publication History

Abstract

In recent years, Graph Neural Networks (GNNs) have been widely used in Collaborative Filtering (CF), one of the most popular methods in recommender systems. However, most existing works focus on designing an individual model architecture given a specific scenario, without studying the influences of different design dimensions. Thus, it remains a challenging problem to quickly obtain a top-performing model in a new recommendation scenario. To address the problem, in this work, we make the first attempt to profile the design space of GNN-based CF methods to enrich the understanding of different design dimensions as well as provide a novel paradigm of model design. Specifically, a unified framework of GNN-based CF is proposed, on top of which a design space is developed and evaluated by extensive experiments. Interesting findings on the impacts of different design dimensions on recommendation performance are obtained. Guided by the empirical findings, we further prune the design space to obtain a compact one containing a higher concentration of top-performing models. Empirical studies demonstrate its high quality and strong generalization ability.

References

[1]

Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263 (2017).

[2]

Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2020. Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach. In Proceedings of the AAAI Conference on Artificial Intelligence. 27--34.

[3]

Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2019. Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In Proceedings of the Conference on Recommender Systems. 101--109.

Digital Library

[4]

Yuhui Ding, Quanming Yao, Huan Zhao, and Tong Zhang. 2021. Diffmg: Differentiable meta graph search for heterogeneous graph neural networks. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 279--288.

Digital Library

[5]

Gideon Dror, Noam Koenigstein, Yehuda Koren, and Markus Weimer. 2012. The yahoo! music dataset and kdd-cup'11. In Proceedings of KDD Cup 2011 . 3--18.

[6]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In Proceedings of the International Conference on World Wide Web. 417--426.

Digital Library

[7]

Yang Gao, Hong Yang, Peng Zhang, Chuan Zhou, and Yue Hu. 2020. Graphnas: Graph neural architecture search with reinforcement learning. In Proceedings of the International Joint Conference on Artificial Intelligence . 1403--1409.

[8]

William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. 1025--1035.

[9]

Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the International Conference on World Wide Web. 507--517.

Digital Library

[10]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval . 639--648.

Digital Library

[11]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the International Conference on World Wide Web. 173--182.

Digital Library

[12]

Jonathan L Herlocker, Joseph A Konstan, Al Borchers, and John Riedl. 1999. An algorithmic framework for performing collaborative filtering. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 230--237.

Digital Library

[13]

Mohsen Jamali and Martin Ester. 2010. A matrix factorization technique with trust propagation for recommendation in social networks. In Proceedings of the Conference on Recommender systems. 135--142.

Digital Library

[14]

Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of International Conference on Learning Representations .

[15]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of International Conference on Learning Representations .

[16]

Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 426--434.

Digital Library

[17]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009), 30--37.

Digital Library

[18]

Joonseok Lee, Seungyeon Kim, Guy Lebanon, and Yoram Singer. 2013. Local low-rank matrix approximation. In Proceedings of International Conference on Machine Learning. 82--90.

[19]

Joonseok Lee, Mingxuan Sun, and Guy Lebanon. 2012. A comparative study of collaborative filtering algorithms. arXiv preprint arXiv:1205.3193 (2012).

[20]

Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, Vol. 7, 1 (2003), 76--80.

[21]

Ilya Loshchilov and Frank Hutter. 2017. Sgdr: Stochastic gradient descent with warm restarts. In Proceedings of International Conference on Learning Representations .

[22]

Qingsong Lv, Ming Ding, Qiang Liu, Yuxiang Chen, Wenzheng Feng, Siming He, Chang Zhou, Jianguo Jiang, Yuxiao Dong, and Jie Tang. 2021. Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining .

[23]

Hao Ma, Dengyong Zhou, Chao Liu, Michael R Lyu, and Irwin King. 2011. Recommender systems with social regularization. In Proceedings of the International Conference on Web Search and Data Mining. 287--296.

Digital Library

[24]

Andriy Mnih and Russ R Salakhutdinov. 2007. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems. 1257--1264.

[25]

Federico Monti, Michael M Bronstein, and Xavier Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks. In Advances in Neural Information Processing Systems. 3697--3707.

[26]

Ilija Radosavovic, Justin Johnson, Saining Xie, Wan-Yen Lo, and Piotr Dollár. 2019. On network design spaces for visual recognition. In Proceedings of the International Conference on Computer Vision. 1882--1890.

[27]

Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. 2020. Designing network design spaces. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 10428--10436.

[28]

Steffen Rendle, Walid Krichene, Li Zhang, and John Anderson. 2020. Neural collaborative filtering vs. matrix factorization revisited. In Proceedings of the Conference on Recommender Systems. 240--248.

Digital Library

[29]

Steffen Rendle, Li Zhang, and Yehuda Koren. 2019. On the difficulty of evaluating baselines: A study on recommender systems. arXiv preprint arXiv:1905.01395 (2019).

[30]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the International Conference on World Wide Web. 285--295.

Digital Library

[31]

Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).

[32]

Zhu Sun, Di Yu, Hui Fang, Jie Yang, Xinghua Qu, Jie Zhang, and Cong Geng. 2020. Are we evaluating rigorously? benchmarking recommendation for reproducible evaluation and fair comparison. In Proceedings of the Conference on Recommender Systems. 23--32.

Digital Library

[33]

Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V Le. 2019. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 2820--2828.

[34]

Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of International Conference on Learning Representations .

[35]

Xin Wang, Shuyi Fan, Kun Kuang, and Wenwu Zhu. 2021. Explainable automated graph representation learning with hyperparameter importance. In Proceedings of International Conference on Machine Learning . 10727--10737.

[36]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval . 165--174.

Digital Library

[37]

Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat-Seng Chua. 2020 a. Disentangled Graph Collaborative Filtering. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval . 1001--1010.

Digital Library

[38]

Xiao Wang, Ruijia Wang, Chuan Shi, Guojie Song, and Qingyong Li. 2020 b. Multi-component graph convolutional collaborative filtering. In Proceedings of the AAAI Conference on Artificial Intelligence. 6267--6274.

[39]

Shiwen Wu, Fei Sun, Wentao Zhang, and Bin Cui. 2020. Graph neural networks in recommender systems: a survey. arXiv preprint arXiv:2011.02260 (2020).

[40]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How powerful are graph neural networks?. In Proceedings of International Conference on Learning Representations .

[41]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 974--983.

Digital Library

[42]

Jiaxuan You, Rex Ying, and Jure Leskovec. 2020. Design Space for Graph Neural Networks. In Advances in Neural Information Processing Systems. 17009--17021.

[43]

Hengrui Zhang and Julian McAuley. 2020. Stacked Mixed-Order Graph Convolutional Networks for Collaborative Filtering. In Proceedings of the International Conference on Data Mining. 73--81.

[44]

Sixiao Zhang, Hongxu Chen, Xiao Ming, Lizhen Cui, Hongzhi Yin, and Guandong Xu. 2021 a. Where are we in embedding spaces? A Comprehensive Analysis on Network Embedding Approaches for Recommender Systems. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining .

[45]

Ziwei Zhang, Xin Wang, and Wenwu Zhu. 2021 b. Automated Machine Learning on Graphs: A Survey. Proceedings of the International Joint Conference on Artificial Intelligence, 4704--4712.

[46]

Huan Zhao, Quanming Yao, James T Kwok, and Dik Lun Lee. 2017. Collaborative filtering with social local models. In Proceedings of the International Conference on Data Mining. 645--654.

[47]

Huan Zhao, Quanming Yao, and Weiwei Tu. 2021 b. Search to aggregate neighborhood for graph neural network. In Proceedings of the International Conference on Data Engineering. 552--563.

[48]

Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Kaiyuan Li, Yushuo Chen, Yujie Lu, Hui Wang, Changxin Tian, Xingyu Pan, et al. 2021 a. RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms. In Proceedings of the International Conference on Information and Knowledge Management .

Digital Library

Cited By

Shen ZGuo SWen YWei LZhang WLuo YWu CYao Q(2025)Searching to extrapolate embedding for out-of-graph node representation learningNeural Networks10.1016/j.neunet.2024.107118184(107118)Online publication date: Apr-2025
https://doi.org/10.1016/j.neunet.2024.107118
Xia JLi DGu HLu TZhang PShang LGu NChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Hierarchical Graph Signal Processing for Collaborative FilteringProceedings of the ACM Web Conference 202410.1145/3589334.3645368(3229-3240)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645368
Hao PLiu SBai C(2024)Intent-Aware Graph-Level Embedding Learning Based RecommendationJournal of Computer Science and Technology10.1007/s11390-024-3522-939:5(1138-1152)Online publication date: 5-Dec-2024
https://doi.org/10.1007/s11390-024-3522-9
Show More Cited By

Index Terms

Profiling the Design Space for Graph Neural Networks based Collaborative Filtering
1. Information systems
  1. Information systems applications
    1. Data mining
      1. Collaborative filtering

Recommendations

Trust-based collaborative filtering: tackling the cold start problem using regular equivalence
RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

User-based Collaborative Filtering (CF) is one of the most popular approaches to create recommender systems. This approach is based on finding the most relevant k users from whose rating history we can extract items to recommend. CF, however, suffers ...
Typicality-Based Collaborative Filtering Recommendation

Collaborative filtering (CF) is an important and popular technology for recommender systems. However, current CF methods suffer from such problems as data sparsity, recommendation inaccuracy, and big-error in predictions. In this paper, we borrow ideas ...
Revisiting tendency based collaborative filtering for personalized recommendations
CODS-COMAD '18: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data

Recommender systems alleviates the problem of information overload by providing personalized suggestions to the users. In this context, recently introduced tendency based recommendation technique is proven to be more simple, intuitive and accurate than ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

February 2022

1690 pages

ISBN:9781450391320

DOI:10.1145/3488560

General Chairs:
K. Selcuk Candan
Arizona State University, USA
,
Huan Liu
Arizona State University, USA
,
Program Chairs:
Leman Akoglu
Carnegie Mellon University, USA
,
Xin Luna Dong
Meta Platforms, Inc. (former Facebook), USA
,
Jiliang Tang
Michigan State University, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

WSDM '22

Sponsor:

WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining

February 21 - 25, 2022

AZ, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
428
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)3

Reflects downloads up to 06 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shen ZGuo SWen YWei LZhang WLuo YWu CYao Q(2025)Searching to extrapolate embedding for out-of-graph node representation learningNeural Networks10.1016/j.neunet.2024.107118184(107118)Online publication date: Apr-2025
https://doi.org/10.1016/j.neunet.2024.107118
Xia JLi DGu HLu TZhang PShang LGu NChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Hierarchical Graph Signal Processing for Collaborative FilteringProceedings of the ACM Web Conference 202410.1145/3589334.3645368(3229-3240)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645368
Hao PLiu SBai C(2024)Intent-Aware Graph-Level Embedding Learning Based RecommendationJournal of Computer Science and Technology10.1007/s11390-024-3522-939:5(1138-1152)Online publication date: 5-Dec-2024
https://doi.org/10.1007/s11390-024-3522-9
Anelli VMalitesta DPomo CBellogin ADi Sciascio EDi Noia T(2023)Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven AnalysisProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3609489(350-361)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3609489
Oh BKim YMin BFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Dual-Oriented Contrast for Recommendation with A Stop-Gradient OperationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614852(1939-1948)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614852
Wen YGao CYi LQiu LWang YLi YSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Efficient and Joint Hyperparameter and Architecture Search for Collaborative FilteringProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599322(2547-2558)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599322
Wei LHe ZZhao HYao Q(2023)Search to Capture Long-range Dependency with Stacking GNNs for Graph ClassificationProceedings of the ACM Web Conference 202310.1145/3543507.3583486(588-598)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583486
Li CXia LRen XYe YXu YHuang CChen HDuh WHuang HKato MMothe JPoblete B(2023)Graph Transformer for RecommendationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591723(1680-1689)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591723
Ren XXia LZhao JYin DHuang CChen HDuh WHuang HKato MMothe JPoblete B(2023)Disentangled Contrastive Collaborative FilteringProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591665(1137-1146)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591665
Bi FHe TXie YLuo X(2023)Two-Stream Graph Convolutional Network-Incorporated Latent Feature AnalysisIEEE Transactions on Services Computing10.1109/TSC.2023.324165916:4(3027-3042)Online publication date: 1-Jul-2023
https://doi.org/10.1109/TSC.2023.3241659
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten