short-paper

An Elementary View on Factorization Machines

Author:

Sebastian PrilloAuthors Info & Claims

RecSys '17: Proceedings of the Eleventh ACM Conference on Recommender Systems

Pages 179 - 183

https://doi.org/10.1145/3109859.3109892

Published: 27 August 2017 Publication History

Get Access

Abstract

Factorization Machines (FMs) are a model class capable of learning pairwise (and in general higher order) feature interactions from high dimensional, sparse data. In this paper we adopt an elementary view on FMs. Specifically, we view FMs as a sum of simple surfaces - a hyperplane plus several squared hyperplanes - in the original feature space. This elementary view, although equivalent to that of low rank matrix factorization, is geometrically more intuitive and points to some interesting generalizations. Led by our intuition, we challenge our understanding of the inductive bias of FMs by showing a simple dataset where FMs counterintuitively fail to learn the weight of the interaction between two features. We discuss the reasons, and mathematically formulate and prove a form of this limitation. Also inspired by our elementary view, we propose modeling intermediate orders of interaction, such as 1.5-way FMs. Beyond the specific proposals, the goal of this paper is to expose our thoughts and ideas to the research community in an effort to take FMs to the next level.

References

[1]

Mathieu Blondel, Akinori Fujino, and Naonori Ueda. 2015. Convex Factorization Machines. In Proceedings, Part II, of the European Conference on Machine Learning and Knowledge Discovery in Databases - Volume 9285 (ECML PKDD 2015). Springer-Verlag New York, Inc., New York, NY, USA, 19--35.

Crossref

Google Scholar

[2]

Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. 2016. Higher-Order Factorization Machines. In Advances in Neural Information Processing Systems 29, D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (Eds.). Curran Associates, Inc., 3351--3359. http://papers.nips.cc/paper/6144-higher-order-factorization-machines.pdf

Google Scholar

[3]

Mathieu Blondel, Masakazu Ishihata, Akinori Fujino, and Naonori Ueda. 2016. Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (ICML'16). JMLR.org, 850--858. http://dl.acm.org/citation.cfm?id=3045390.3045481

Digital Library

Google Scholar

[4]

F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5, 4, Article 19 (Dec. 2015), 19 pages.

Digital Library

Google Scholar

[5]

Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware Factorization Machines for CTR Prediction. In Proceedings of the 10th ACM Conference on Recommender Systems.

Digital Library

Google Scholar

[6]

Steffen Rendle. 2010. Factorization Machines. In Proceedings of the 2010 IEEE International Conference on Data Mining.

Digital Library

Google Scholar

[7]

Steffen Rendle. 2012. Factorization Machines with libFM. ACM Trans. Intell. Syst. Technol. 3 (2012).

Digital Library

Google Scholar

Cited By

View all

Lyu FTang XGuo HTang RHe XZhang RLiu X(2022)Memorize, Factorize, or be Naive: Learning Optimal Feature Interaction Methods for CTR Prediction2022 IEEE 38th International Conference on Data Engineering (ICDE)10.1109/ICDE53745.2022.00113(1450-1462)Online publication date: May-2022
https://doi.org/10.1109/ICDE53745.2022.00113
Liang YWang QXiong KZheng XYu ZZeng D(2021)Robust Detection of Malicious URLs with Self-Paced Wide & Deep LearningIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2021.3121388(1-1)Online publication date: 2021
https://doi.org/10.1109/TDSC.2021.3121388
Han DChen YZhang S(2021)Implicit social recommendation algorithm based on multilayer fuzzy perception similarityInternational Journal of Machine Learning and Cybernetics10.1007/s13042-021-01409-2Online publication date: 17-Aug-2021
https://doi.org/10.1007/s13042-021-01409-2
Show More Cited By

Index Terms

An Elementary View on Factorization Machines
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Factorization methods
      2. Kernel methods
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Convex factorization machines
ECMLPKDD'15: Proceedings of the 2015th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II

Factorization machines are a generic framework which allows to mimic many factorization models simply by feature engineering. In this way, they combine the high predictive accuracy of factorization models with the flexibility of feature engineering. ...
Factorization machines with regularization for sparse feature interactions

Factorization machines (FMs) are machine learning predictive models based on second-order feature interactions and FMs with sparse regularization are called sparse FMs. Such regularizations enable feature selection, which selects the most relevant ...
Gradient boosting factorization machines
RecSys '14: Proceedings of the 8th ACM Conference on Recommender systems

Recommendation techniques have been well developed in the past decades. Most of them build models only based on user item rating matrix. However, in real world, there is plenty of auxiliary information available in recommendation systems. We can utilize ...

Comments

Information & Contributors

Information

Published In

RecSys '17: Proceedings of the Eleventh ACM Conference on Recommender Systems

August 2017

466 pages

ISBN:9781450346528

DOI:10.1145/3109859

General Chairs:
Paolo Cremonesi
Politecnico di Milano, Italy
,
Francesco Ricci
Free University Bozen-Bolzano, Italy
,
Program Chairs:
Shlomo Berkovsky
CSIRO, Australia
,
Alexander Tuzhilin
New York University, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

RecSys '17

Sponsor:

SIGCHI

RecSys '17: Eleventh ACM Conference on Recommender Systems

August 27 - 31, 2017

Como, Italy

Acceptance Rates

RecSys '17 Paper Acceptance Rate 26 of 125 submissions, 21%;

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
610
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)2

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lyu FTang XGuo HTang RHe XZhang RLiu X(2022)Memorize, Factorize, or be Naive: Learning Optimal Feature Interaction Methods for CTR Prediction2022 IEEE 38th International Conference on Data Engineering (ICDE)10.1109/ICDE53745.2022.00113(1450-1462)Online publication date: May-2022
https://doi.org/10.1109/ICDE53745.2022.00113
Liang YWang QXiong KZheng XYu ZZeng D(2021)Robust Detection of Malicious URLs with Self-Paced Wide & Deep LearningIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2021.3121388(1-1)Online publication date: 2021
https://doi.org/10.1109/TDSC.2021.3121388
Han DChen YZhang S(2021)Implicit social recommendation algorithm based on multilayer fuzzy perception similarityInternational Journal of Machine Learning and Cybernetics10.1007/s13042-021-01409-2Online publication date: 17-Aug-2021
https://doi.org/10.1007/s13042-021-01409-2
Chang CTang XYuan BHsia JLiu ZDong ZHe XLin C(2020)AutoConjunction: Adaptive Model-based Feature Conjunction for CTR Prediction2020 21st IEEE International Conference on Mobile Data Management (MDM)10.1109/MDM48529.2020.00043(202-209)Online publication date: Jun-2020
https://doi.org/10.1109/MDM48529.2020.00043
Guo WTang RGuo HHan JYang WZhang YPiwowarski BChevalier MGaussier EMaarek YNie JScholer F(2019)Order-aware Embedding Neural Network for CTR PredictionProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3331184.3331332(1121-1124)Online publication date: 18-Jul-2019
https://dl.acm.org/doi/10.1145/3331184.3331332
da Silva ARodrigues Lde Oliveira Rech LLuiz A(2019)RDFM: Resilient Distributed Factorization MachinesArtificial Intelligence and Soft Computing10.1007/978-3-030-20915-5_52(585-594)Online publication date: 27-May-2019
https://doi.org/10.1007/978-3-030-20915-5_52

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Convex factorization machines

Factorization machines with regularization for sparse feature interactions

Gradient boosting factorization machines