research-article

Recommender System Using Sequential and Global Preference via Attention Mechanism and Topic Modeling

Authors:

Kyeongpil Kang,

Jaegul ChooAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 1543 - 1552

https://doi.org/10.1145/3357384.3358054

Published: 03 November 2019 Publication History

Abstract

Deep neural networks improved the accuracy of sequential recommendation approach which takes into account the sequential patterns of user logs, e.g., a purchase history of a user. However, incorporating only the individual's recent logs may not be sufficient in properly reflecting global preferences and trends across all users and items. In response, we propose a self-attentive sequential recommender system with topic modeling-based category embedding as a novel approach to exploit global information in the process of sequential recommendation. Our self-attention module effectively leverages the sequential patterns from the user's recent history. In addition, our novel category embedding approach, which utilizes the information computed by topic modeling, efficiently captures global information that the user generally prefers. Furthermore, to provide diverse recommendations as well as to prevent overfitting, our model also incorporates a vector obtained by random sampling. Experimental studies show that our model outperforms state-of-the-art sequential recommendation models, and that category embedding effectively provides global preference information.

References

[1]

Rakesh Agrawal, Tomasz Imieli'nski, and Arun Swami. 1993. Mining association rules between sets of items in large databases. In ACM SIGMOD . 207--216.

[2]

R. Agrawal and R. Srikant. 1995. Mining sequential patterns. In ICDE. 3--14.

[3]

David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent Dirichlet allocation. JMLR (2003), 993--1022.

Digital Library

[4]

Chenwei Cai, Ruining He, and Julian McAuley. 2017. SPMC: socially-aware personalized markov chains for sparse sequential recommendation. In IJCAI . 1476--1482.

[5]

Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, and Diana Inkpen. 2017. Natural language inference with external knowledge. arXiv:1711.04289 (2017).

[6]

Wen-Yen Chen, Dong Zhang, and Edward Y Chang. 2008. Combinational collaborative filtering for personalized community recommendation. In KDD . 115--123.

[7]

Disheng Dong, Xiaolin Zheng, Ruixun Zhang, and Yan Wang. 2018. Recurrent Collaborative Filtering for Unifying General and Sequential Recommender. In IJCAI . 3350--3356.

[8]

Songjie Gong. 2010. A collaborative filtering recommendation algorithm based on user clustering and item clustering. Journal of Software (2010), 745--752.

[9]

Ruining He, Wang-Cheng Kang, and Julian McAuley. 2017. Translation-based recommendation. In RecSys. 161--169.

[10]

Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In ICDM . 191--200.

[11]

Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat-Seng Chua. 2018. Outer product-based neural collaborative filtering. In IJCAI . 2227--2233.

[12]

Balázs Hidasi and Alexandros Karatzoglou. 2018. Recurrent neural networks with top-k gains for session-based recommendations. In CIKM . 843--852.

[13]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv:1511.06939 (2015).

[14]

Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recommendation. In ICDM. 197--206.

[15]

Tero Karras, Samuli Laine, and Timo Aila. 2018. A style-based generator architecture for generative adversarial networks. arXiv:1812.04948 (2018).

[16]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980 (2014).

[17]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv:1312.6114 (2013).

[18]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. IEEE Computer Society (2009), 30--37.

Digital Library

[19]

Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In ICML. 1188--1196.

[20]

Daniel D Lee and H Sebastian Seung. 2001. Algorithms for non-negative matrix factorization. In NIPS. 556--562.

[21]

Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In CIKM. 1419--1428.

[22]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. JMLR (2008), 2579--2605.

[23]

Benjamin M Marlin. 2004. Modeling user rating profiles for collaborative filtering. In NIPS. 627--634.

[24]

Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426 (2018).

[25]

Lei Mei, Pengjie Ren, Zhumin Chen, Liqiang Nie, Jun Ma, and Jian-Yun Nie. 2018. An attentive interaction network for context-aware recommendations. In CIKM . 157--166.

[26]

Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. 2016. Natural Language Inference by Tree-Based Convolution and Heuristic Matching. In ACL . 130--136.

[27]

Shuzi Niu and Rongzhi Zhang. 2017. Collaborative sequence prediction for sequential recommender. In CIKM . 2239--2242.

[28]

Rong Pan, Yunhong Zhou, Bin Cao, Nathan N Liu, Rajan Lukose, Martin Scholz, and Qiang Yang. 2008. One-class collaborative filtering. In ICDM. 502--511.

[29]

Wenjie Pei, Jie Yang, Zhu Sun, Jie Zhang, Alessandro Bozzon, and David MJ Tax. 2017. Interacting attention-gated recurrent networks for recommendation. In CIKM . 1459--1468.

[30]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In UAI. 452--461.

Digital Library

[31]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW . 811--820.

[32]

Hanhuai Shan and Arindam Banerjee. 2010. Generalized probabilistic matrix factorizations for collaborative filtering. In ICDM . 1025--1030.

[33]

Shaoyun Shi, Min Zhang, Yiqun Liu, and Shaoping Ma. 2018. Attention-based Adaptive Model to Unify Warm and Cold Starts Recommendation. In CIKM . 127--136.

[34]

Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In ICWSM . 565--573.

[35]

Yee W Teh, Michael I Jordan, Matthew J Beal, and David M Blei. 2005. Sharing clusters among related groups: Hierarchical Dirichlet processes. In NIPS . 1385--1392.

[36]

Lyle H Ungar and Dean P Foster. 1998. Clustering methods for collaborative filtering. In AAAI workshop on recommendation systems . 114--129.

[37]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. 5998--6008.

[38]

Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning for recommender systems. In KDD. 1235--1244.

[39]

Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019. Knowledge Graph Convolutional Networks for Recommender Systems. In WWW . 3307--3313.

[40]

Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017. Recurrent recommender networks. In ICDM. 495--503.

[41]

Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for top-n recommender systems. In ICWSM . 153--162.

[42]

Haochao Ying, Fuzhen Zhuang, Fuzheng Zhang, Yanchi Liu, Guandong Xu, Xing Xie, Hui Xiong, and Jian Wu. 2018. Sequential recommender system based on hierarchical attention networks. In IJCAI . 3926--3932.

[43]

Chenyi Zhang, Ke Wang, Hongkun Yu, Jianling Sun, and Ee-Peng Lim. 2014. Latent factor transition for dynamic collaborative filtering. In SDM. 452--460.

[44]

Qi Zhang, Jiawen Wang, Haoran Huang, Xuanjing Huang, and Yeyun Gong. 2017. Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network. In IJCAI . 3420--3426.

[45]

Shuai Zhang, Yi Tay, Lina Yao, and Aixin Sun. 2018. Next Item Recommendation with Self-Attention. arXiv:1808.06414 (2018).

[46]

Ziwei Zhu, Xia Hu, and James Caverlee. 2018. Fairness-aware tensor-based recommendation. In CIKM. 1153--1162.

Cited By

Zhang PYan YZhang XLi CWang SHuang FKim SHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657721(1285-1295)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657721
Jiang JZhang PLuo YLi CKim JZhang KWang SXie XKim SFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614773(976-986)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614773
Liu SWang BDeng XYang L(2022)Self-Attentive Graph Convolution Network With Latent Group Mining and Collaborative Filtering for Personalized RecommendationIEEE Transactions on Network Science and Engineering10.1109/TNSE.2021.31106779:5(3212-3221)Online publication date: 1-Sep-2022
https://doi.org/10.1109/TNSE.2021.3110677
Show More Cited By

Index Terms

Recommender System Using Sequential and Global Preference via Attention Mechanism and Topic Modeling
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Topic modeling
2. Information systems
  1. Information systems applications
    1. Data mining
      1. Collaborative filtering

Recommendations

Improving Accuracy of Recommender System by Item Clustering

Recommender System (RS) predicts user's ratings towards items, and then recommends highly-predicted items to user. In recent years, RS has been playing more and more important role in the agent research field. There have been a great deal of researches ...
A novel temporal recommender system based on multiple transitions in user preference drift and topic review evolution
Abstract
Recommender systems are challenging research problems being exploited to suggest new items or services, such as books, music and movies, and even people, to users based on information about the user profile or the recommended items. To ...
Highlights
- A novel temporal multitransition in user preference drift systems was proposed.
A New Approach for Recommender System
ICACS '17: Proceedings of the 1st International Conference on Algorithms, Computing and Systems

In today's e-commerce environment, Collaborative Filtering (CF) is a widely used algorithm for recommender system, which is to identify the users who have similar preferences to the target user, and to predict the preference of the target user according ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Research Foundation of Korea

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
782
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)9

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang PYan YZhang XLi CWang SHuang FKim SHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657721(1285-1295)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657721
Jiang JZhang PLuo YLi CKim JZhang KWang SXie XKim SFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614773(976-986)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614773
Liu SWang BDeng XYang L(2022)Self-Attentive Graph Convolution Network With Latent Group Mining and Collaborative Filtering for Personalized RecommendationIEEE Transactions on Network Science and Engineering10.1109/TNSE.2021.31106779:5(3212-3221)Online publication date: 1-Sep-2022
https://doi.org/10.1109/TNSE.2021.3110677
Li YYue SZheng JWang W(2022)Customer-oriented product design: an integrated decision framework with sentiment analysis and optimisation modelJournal of Control and Decision10.1080/23307706.2022.214660711:2(165-179)Online publication date: 21-Nov-2022
https://doi.org/10.1080/23307706.2022.2146607
Khan NMa ZUllah APolat K(2022)Similarity attributed knowledge graph embedding enhancement for item recommendationInformation Sciences: an International Journal10.1016/j.ins.2022.08.124613:C(69-95)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.ins.2022.08.124
Wang CChen BChiang J(2021)TDD-BPR: The topic diversity discovering on Bayesian personalized ranking for personalized recommender systemNeurocomputing10.1016/j.neucom.2021.02.016441(202-213)Online publication date: Jun-2021
https://doi.org/10.1016/j.neucom.2021.02.016
Thaipisutikul TShih T(2021)A novel context-aware recommender system based on a deep sequential learning approach (CReS)Neural Computing and Applications10.1007/s00521-020-05640-w33:17(11067-11090)Online publication date: 3-Jan-2021
https://doi.org/10.1007/s00521-020-05640-w
Ahn SKo HKang J(2021)Regularized Categorical Embedding for Effective Demand Forecasting of Bike Sharing SystemData Science and Digital Transformation in the Fourth Industrial Revolution10.1007/978-3-030-64769-8_14(179-193)Online publication date: 3-Jan-2021
https://doi.org/10.1007/978-3-030-64769-8_14

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents