research-article

Item Recommendation with Variational Autoencoders and Heterogeneous Priors

Authors:

Giannis Karamanolakis,

Kevin Raji Cherian,

Ananth Ravi Narayan,

Tony JebaraAuthors Info & Claims

DLRS 2018: Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems

Pages 10 - 14

https://doi.org/10.1145/3270323.3270329

Published: 06 October 2018 Publication History

Abstract

In recent years, Variational Autoencoders (VAEs) have been shown to be highly effective in both standard collaborative filtering applications and extensions such as incorporation of implicit feedback. We extend VAEs to collaborative filtering with side information, for instance when ratings are combined with explicit text feedback from the user. Instead of using a user-agnostic standard Gaussian prior, we incorporate user-dependent priors in the latent VAE space to encode users' preferences as functions of the review text. Taking into account both the rating and the text information to represent users in this multimodal latent space is promising to improve recommendation quality. Our proposed model is shown to outperform the existing VAE models for collaborative filtering (up to 29.41% relative improvement in ranking metric) along with other baselines that incorporate both user ratings and text for item recommendation.

References

[1]

Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2016. A simple but tough-to-beat baseline for sentence embeddings. (2016).

[2]

Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research 3, Feb (2003), 1137--1155.

[3]

David M Blei and John D Lafferty. 2009. Topic models. Text mining: classification, clustering, and applications 10, 71 (2009), 34.

[4]

David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993--1022.

Digital Library

[5]

Samuel Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating Sentences from a Continuous Space. In Proceedings of the Twentieth Conference on Computational Natural Language Learning (CoNLL).

[6]

Li Chen, Guanliang Chen, and Feng Wang. 2015. Recommender systems based on user reviews: the state of the art. User Modeling and User-Adapted Interaction 25, 2(2015), 99--154.

Digital Library

[7]

Minmin Chen. 2017. Efficient vector representation for documents through corruption. International Conference on Learning Representations 2017 (2017).

[8]

Xi Chen, Diederik P Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Variational lossy autoencoder. arXiv preprint arXiv:1611.02731 (2016).

[9]

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 7--10.

Digital Library

[10]

Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning. ACM, 160--167.

Digital Library

[11]

Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 191--198.

Digital Library

[12]

Qiming Diao, Minghui Qiu, Chao-Yuan Wu, Alexander J Smola, Jing Jiang, and Chong Wang. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS). In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 193--202.

Digital Library

[13]

Samuel Gershman and Noah Goodman. 2014. Amortized inference in probabilistic reasoning. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 36.

[14]

Prem K Gopalan, Laurent Charlin, and David Blei. 2014. Content-based recommendations with poisson factorization. In Advances in Neural Information Processing Systems. 3176--3184.

Digital Library

[15]

Prasoon Goyal, Zhiting Hu, Xiaodan Liang, Chenyu Wang, Eric P Xing, and Carnegie Mellon. 2017. Nonparametric Variational Auto-Encoders for Hierarchical Representation Learning. In 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 5104--5112.

[16]

Matthew D Hoffman and Matthew J Johnson. 2016. Elbo surgery: yet another way to carve up the variational evidence lower bound. In Workshop in Advances in Approximate Bayesian Inference, NIPS.

[17]

Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Vol. 1. 1681--1691.

[18]

Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv: 1611.01144 (2016).

[19]

Dietmar Jannach and Malte Ludewig. 2017. When recurrent neural networks meet the neighborhood for session-based recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 306--310.

Digital Library

[20]

Yoon Kim, Sam Wiseman, Andrew C Miller, David Sontag, and Alexander M Rush. 2018. Semi-Amortized Variational Autoencoders. arXiv preprint arXiv:1802.02550 (2018).

[21]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013. (2013).

[22]

Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International Conference on Machine Learning. 1188--1196.

Digital Library

[23]

Xiaopeng Li and James She. 2017. Collaborative variational autoencoder for recommender systems. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 305--314.

Digital Library

[24]

Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. arXiv preprint arXiv: 1802.05814 (2018).

[25]

Julian McAuley and Jure Leskovec. 2013. Hidden factors and hidden topics: understanding rating dimensions with review text. In Proceedings of the 7th ACM conference on Recommender systems. ACM, 165--172.

Digital Library

[26]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv: 1301.3781 (2013).

[27]

Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013. Exploiting similarities among languages for machine translation. CoRR, abs/1309.4168 (2013).

[28]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.

Digital Library

[29]

Cataldo Musto, Marco de Gemmis, Giovanni Semeraro, and Pasquale Lops. 2017. A Multi-criteria Recommender System Exploiting Aspect-based Sentiment Analysis of Users' Reviews. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 321--325.

Digital Library

[30]

Eric Nalisnick, Lars Hertel, and Padhraic Smyth. 2016. Approximate inference for deep latent gaussian mixtures. In NIPS Workshop on Bayesian Deep Learning, Vol. 2.

[31]

Eric Nalisnick and Padhraic Smyth. 2017. Stick-breaking variational autoencoders. In International Conference on Learning Representations (ICLR).

[32]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532--1543.

[33]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In International Conference on Machine Learning. 1278--1286.

Digital Library

[34]

Elena Smirnova and Flavian Vasile. 2017. Contextual sequence modeling for recommendation with recurrent neural networks. In Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems. ACM, 2--9.

Digital Library

[35]

Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther. 2016. Ladder variational autoencoders. In Advances in neural information processing systems. 3738--3746.

[36]

Aaron Van den Oord, Sander Dieleman, and Benjamin Schrauwen. 2013. Deep content-based music recommendation. In Advances in neural information processing systems. 2643--2651.

Digital Library

[37]

Chong Wang and David M Blei. 2011. Collaborative topic modeling for recommending scientific articles. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 448--456.

Digital Library

[38]

Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1235--1244.

Digital Library

[39]

Xinxi Wang and Ye Wang. 2014. Improving content-based and hybrid music recommendation using deep learning. In Proceedings of the 22nd ACM international conference on Multimedia. ACM, 627--636.

Digital Library

[40]

John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. Towards universal paraphrastic sentence embeddings. In Proceedings of the International Conference on Learning Representations (ICLR) (2015) (2015).

[41]

Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for top-n recommender systems. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining. ACM, 153--162.

Digital Library

[42]

Shuang-Hong Yang, Bo Long, Alexander J Smola, Hongyuan Zha, and Zhaohui Zheng. 2011. Collaborative competitive filtering: learning recommender using context of user choice. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. ACM, 295--304.

Digital Library

[43]

Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick. 2017. Improved Variational Autoencoders for Text Modeling using Dilated Convolutions. In International Conference on Machine Learning. 3881--3890.

Cited By

Aljunid MD.H. MHooshmand MAli WShetty AAlzoubah S(2025)A collaborative filtering recommender systems: SurveyNeurocomputing10.1016/j.neucom.2024.128718617(128718)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.128718
Liu WU LLiang SZhu HYu JLiu YYin J(2024)VAE*: A Novel Variational Autoencoder via Revisiting Positive and Negative Samples for Top-N RecommendationACM Transactions on Knowledge Discovery from Data10.1145/368055218:9(1-24)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3680552
Liang SPan Zliu wYin Jde Rijke M(2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3663364
Show More Cited By

Index Terms

Item Recommendation with Variational Autoencoders and Heterogeneous Priors
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Item recommendation in collaborative tagging systems via heuristic data fusion

Collaborative tagging systems have been popular on the Web. However, information overload results in the increasing need for recommender services from users, and thus item recommendation has been one of the key issues in such systems. In this paper, we ...
Augmented Variational Autoencoders for Collaborative Filtering with Auxiliary Information
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Recommender systems offer critical services in the age of mass information. A good recommender system selects a certain item for a specific user by recognizing why the user might like the item. This awareness implies that the system should model the ...
Item recommendation based on context-aware model for personalized u-healthcare service

A personalized service in the ubiquitous environment is to provide services or items, which reflect personal tastes, attitudes, and contexts. It is impossible to reflect the context information generated in u-healthcare environments due to the existing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

DLRS 2018: Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems

October 2018

35 pages

ISBN:9781450366175

DOI:10.1145/3270323

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Intuit: Intuit Inc.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

DLRS 2018

DLRS 2018: 3rd Workshop on Deep Learning for Recommender Systems

October 6, 2018

BC, Vancouver, Canada

Acceptance Rates

DLRS 2018 Paper Acceptance Rate 4 of 11 submissions, 36%;

Overall Acceptance Rate 11 of 27 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
500
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Aljunid MD.H. MHooshmand MAli WShetty AAlzoubah S(2025)A collaborative filtering recommender systems: SurveyNeurocomputing10.1016/j.neucom.2024.128718617(128718)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.128718
Liu WU LLiang SZhu HYu JLiu YYin J(2024)VAE*: A Novel Variational Autoencoder via Revisiting Positive and Negative Samples for Top-N RecommendationACM Transactions on Knowledge Discovery from Data10.1145/368055218:9(1-24)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3680552
Liang SPan Zliu wYin Jde Rijke M(2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3663364
Austin DKorikov AToroghi ASanner S(2024)Bayesian Optimization with LLM-Based Acquisition Functions for Natural Language Preference ElicitationProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688142(74-83)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688142
Li NGuo BLiu YDing YYao LFan XYu Z(2024)Hierarchical Constrained Variational Autoencoder for interaction-sparse recommendationsInformation Processing & Management10.1016/j.ipm.2024.10364161:3(103641)Online publication date: May-2024
https://doi.org/10.1016/j.ipm.2024.103641
Yi JZhu YXie JChen Z(2023)Cross-Modal Variational Auto-Encoder for Content-Based Micro-Video Background Music RecommendationIEEE Transactions on Multimedia10.1109/TMM.2021.312825425(515-528)Online publication date: 2023
https://doi.org/10.1109/TMM.2021.3128254
Khadir MHashmi M(2023)High Efficient Quality of Video Compression using Variational Autoencoders in Deep Learning2023 5th International Conference on Power, Control & Embedded Systems (ICPCES)10.1109/ICPCES57104.2023.10075897(1-6)Online publication date: 6-Jan-2023
https://doi.org/10.1109/ICPCES57104.2023.10075897
Feng JLiu MLiang XNie T(2023)ALCRKnowledge-Based Systems10.1016/j.knosys.2023.110829278:COnline publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110829
Feng JLiu MHong SSong S(2023)A Novel Variational Autoencoder with Multi-position Latent Self-attention and Actor-Critic for RecommendationAdvanced Data Mining and Applications10.1007/978-3-031-46661-8_11(155-167)Online publication date: 5-Nov-2023
https://doi.org/10.1007/978-3-031-46661-8_11
Zhang YZhao CYuan MChen MLiu X(2022)Unifying attentive sparse autoencoder with neural collaborative filtering for recommendationIntelligent Data Analysis10.3233/IDA-21604926:4(841-857)Online publication date: 11-Jul-2022
https://doi.org/10.3233/IDA-216049
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents