skip to main content
10.1145/3626772.3657749acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article
Open access

Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender Systems

Published: 11 July 2024 Publication History

Abstract

Two typical forms of bias in user interaction data with recommender systems (RSs) are popularity bias and positivity bias, which manifest themselves as the over-representation of interactions with popular items or items that users prefer, respectively. Debiasing methods aim to mitigate the effect of selection bias on the evaluation and optimization of RSs. However, existing debiasing methods only consider single-factor forms of bias, e.g., only the item (popularity) or only the rating value (positivity). This is in stark contrast with the real world where user selections are generally affected by multiple factors at once. In this work, we consider multifactorial selection bias in RSs. Our focus is on selection bias affected by both item and rating value factors, which is a generalization and combination of popularity and positivity bias. While the concept of multifactorial bias is intuitive, it brings a severe practical challenge as it requires substantially more data for accurate bias estimation. As a solution, we propose smoothing and alternating gradient descent techniques to reduce variance and improve the robustness of its optimization. Our experimental results reveal that, with our proposed techniques, multifactorial bias corrections are more effective and robust than single-factor counterparts on real-world and synthetic datasets.

References

[1]
Himan Abdollahpouri. 2019. Popularity bias in ranking and recommendation. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 529--530.
[2]
Himan Abdollahpouri and Masoud Mansoury. 2020. Multi-sided exposure bias in recommendation. arXiv preprint arXiv:2006.15772 (2020).
[3]
Panagiotis Adamopoulos and Alexander Tuzhilin. 2014. On Over-specialization and Concentration Bias of Recommendations: Probabilistic Neighborhood Selection in Collaborative Filtering Systems. In ACM Recsys. 153--160.
[4]
Roy F Baumeister, Ellen Bratslavsky, Catrin Finkenauer, and Kathleen D Vohs. 2001. Bad is stronger than good. Review of general psychology, Vol. 5, 4 (2001), 323--370.
[5]
Alejandro Bellog'in, Pablo Castells, and Iván Cantador. 2017. Statistical Biases in Information Retrieval Metrics for Recommender Systems. Information Retrieval Journal, Vol. 20 (2017), 606--634.
[6]
Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. 2013. Recommender Systems Survey. Knowledge-based systems, Vol. 46 (2013), 109--132.
[7]
Robin Burke. 2002. Hybrid Recommender Systems: Survey and Experiments. User Modeling and User-adapted Interaction, Vol. 12 (2002), 331--370.
[8]
Roc'io Ca namares and Pablo Castells. 2018. Should I Follow the Crowd? A Probabilistic Analysis of the Effectiveness of Popularity in Recommender Systems. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 415--424.
[9]
Òscar Celma and Pedro Cano. 2008. From hits to niches? or how popular artists can bias music recommendation and discovery. In Proceedings of the 2nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition. 1--8.
[10]
Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023. Bias and Debias in Recommender System: A Survey and Future Directions. ACM Transactions on Information Systems, Vol. 41, 3 (2023), 1--39.
[11]
Minmin Chen, Alex Beutel, Paul Covington, Sagar Jain, Francois Belletti, and Ed H Chi. 2019. Top-k Off-policy Correction for a REINFORCE Recommender System. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. 456--464.
[12]
Joao Felipe Guedes da Silva, Natanael Nunes de Moura Junior, and Luiz Pereira Caloba. 2018. Effects of Data Sparsity on Recommender Systems Based on Collaborative Filtering. In 2018 International Joint Conference on Neural Networks (IJCNN). IEEE, 1--8.
[13]
Thomas J DiCiccio and Bradley Efron. 1996. Bootstrap confidence intervals. Statistical science, Vol. 11, 3 (1996), 189--228.
[14]
Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All the Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency. PMLR, 172--186.
[15]
William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. The Journal of Machine Learning Research, Vol. 23, 1 (2022), 5232--5270.
[16]
Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. 2022. KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 540--550.
[17]
Jin Huang, Harrie Oosterhuis, and Maarten de Rijke. 2022. It Is Different When Items are Older: Debiasing Recommendations When Selection Bias and User Preferences are Dynamic. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 381--389.
[18]
Jin Huang, Harrie Oosterhuis, Maarten de Rijke, and Herke van Hoof. 2020. Keeping Dataset Biases out of the Simulation: A Debiased Simulator for Reinforcement Learning based Recommender Systems. In Proceedings of the 14th ACM Conference on Recommender Systems. 190--199.
[19]
Nouhaila Idrissi, Ahmed Zellou, and Zohra Bakkoury. 2023. “Guess Why I Didn't Rate It”: A New Preference-based Model for Enhanced Top-K Recommendation. International Journal of Intelligent Engineering and Systems, Vol. 16, 3 (2023), 542--551.
[20]
Guido W. Imbens and Donald B. Rubin. 2015. Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge University Press.
[21]
Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased Learning-to-Rank with Biased Feedback. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. 781--789.
[22]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations.
[23]
Norman Knyazev and Harrie Oosterhuis. 2022. The Bandwagon Effect: Not Just Another Bias. In Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval. 243--253.
[24]
Sanjay Krishnan, Jay Patel, Michael J. Franklin, and Ken Goldberg. 2014. A Methodology for Learning, Analyzing, and Mitigating Social Influence Bias in Recommender Systems. In Proceedings of the 8th ACM Conference on Recommender systems. 137--144.
[25]
Haoxuan Li, Yanghao Xiao, Chunyuan Zheng, Peng Wu, and Peng Cui. 2023. Propensity matters: Measuring and enhancing balancing for recommendation. In International Conference on Machine Learning. PMLR, 20182--20194.
[26]
Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 world wide web conference. 689--698.
[27]
Zinan Lin, Dugang Liu, Weike Pan, Qiang Yang, and Zhong Ming. 2023. Transfer learning for collaborative recommendation with biased and unbiased data. Artificial Intelligence, Vol. 324 (2023), 103992.
[28]
Dugang Liu, Pengxiang Cheng, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. 2020. A general knowledge distillation framework for counterfactual recommendation via uniform data. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 831--840.
[29]
Zhongzhou Liu, Yuan Fang, and Min Wu. 2024. Estimating Propensity for Causality-based Recommendation without Exposure Data. Advances in Neural Information Processing Systems, Vol. 36 (2024).
[30]
Christopher D. Manning, Raghavan Prabhakar, and Schütze Hinrich. 2008. Introduction to Information Retrieval. Cambridge University Press.
[31]
Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020. Feedback loop and bias amplification in recommender systems. In Proceedings of the 29th ACM international conference on information & knowledge management. 2145--2148.
[32]
Benjamin M. Marlin and Richard S. Zemel. 2009. Collaborative Prediction and Ranking with Non-random Missing Data. In Proceedings of the Third ACM Conference on Recommender Systems. ACM, 5--12.
[33]
Paolo Massa and Paolo Avesani. 2007. Trust-aware recommender systems. In Proceedings of the 2007 ACM conference on Recommender systems. 17--24.
[34]
Daniel F McCaffrey, Greg Ridgeway, and Andrew R Morral. 2004. Propensity Score Estimation with Boosted Regression for Evaluating Causal Effects in Observational Studies. Psychological Methods, Vol. 9, 4 (2004), 403.
[35]
Tien T. Nguyen, Pik-Mai Hui, F. Maxwell Harper, Loren Terveen, and Joseph A. Konstan. 2014. Exploring the Filter Bubble: The Effect of Using Recommender Systems on Content Diversity. In Proceedings of the 23rd International Conference on World Wide Web. 677--686.
[36]
Harrie Oosterhuis. 2023. Doubly Robust Estimation for Correcting Position Bias in Click Feedback for Unbiased Learning to Rank. ACM Transactions on Information Systems, Vol. 41, 3 (2023), 1--33.
[37]
Zohreh Ovaisi, Ragib Ahsan, Yifan Zhang, Kathryn Vasilaky, and Elena Zheleva. 2020. Correcting for Selection Bias in Learning-to-Rank Systems. In Proceedings of The Web Conference 2020. 1863--1873.
[38]
Umberto Panniello, Shawndra Hill, and Michele Gorgoglione. 2016. The Impact of Profit Incentives on the Relevance of Online Recommendations. Electronic Commerce Research and Applications, Vol. 20 (2016), 87--104.
[39]
Eli Pariser. 2011. The Filter Bubble: How the New Personalized Web is Changing What We Read and How We Think. Penguin.
[40]
Kunwoo Park, Meeyoung Cha, and Eunhee Rhim. 2018. Positivity bias in customer satisfaction ratings. In Companion Proceedings of the The Web Conference 2018. 631--638.
[41]
Bruno Pradel, Nicolas Usunier, and Patrick Gallinari. 2012. Ranking with Non-random Missing Ratings: Influence of Popularity and Positivity on Evaluation Metrics. In Proceedings of the Sixth ACM Conference on Recommender Systems. 147--154.
[42]
Francesco Ricci, Lior Rokach, and Bracha Shapira. 2010. Introduction to Recommender Systems Handbook. In Recommender Systems Handbook. Springer, 1--35.
[43]
Francesco Ricci, Lior Rokach, and Bracha Shapira. 2015. Recommender Systems: Introduction and Challenges. Recommender Systems Handbook (2015), 1--34.
[44]
Yuta Saito. 2020. Asymmetric tri-training for debiasing missing-not-at-random explicit feedback. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 309--318.
[45]
Yuta Saito, Hayato Sakata, and Kazuhide Nakata. 2019. Doubly Robust Prediction and Evaluation Methods Improve Uplift modeling for Observational Data. In Proceedings of the 2019 SIAM International Conference on Data Mining. SIAM, 468--476.
[46]
Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased Recommender Learning from Missing-not-at-random Implicit Feedback. In Proceedings of the 13th International Conference on Web Search and Data Mining. 501--509.
[47]
Fatemeh Sarvi, Ali Vardasbi, Mohammad Aliannejadi, Sebastian Schelter, and Maarten de Rijke. 2023. On the Impact of Outlier Bias on User Clicks. In SIGIR 2023: 46th international ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 18--27.
[48]
Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learning and Evaluation. In International Conference on Machine Learning. PMLR, 1670--1679.
[49]
Harald Steck. 2011. Item Popularity and Recommendation Accuracy. In Proceedings of the Fifth ACM Conference on Recommender Systems. 125--132.
[50]
Harald Steck. 2013. Evaluation of Recommendations: Rating-prediction and Ranking. In Proceedings of the 7th ACM Conference on Recommender Systems. 213--220.
[51]
Alex Strehl, John Langford, Lihong Li, and Sham M Kakade. 2010. Learning from Logged Implicit Exploration Data. In Advances in Neural Information Processing Systems, Vol. 23.
[52]
Adith Swaminathan and Thorsten Joachims. 2015. The Self-normalized Estimator for Counterfactual Learning. In Advances in Neural Information Processing Systems, Vol. 28.
[53]
Gábor Takács and Domonkos Tikk. 2012. Alternating Least Squares for Personalized Ranking. In Proceedings of the Sixth ACM Conference on Recommender Systems. 83--90.
[54]
Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork. 2016. Learning to rank with selection bias in personal search. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 115--124.
[55]
Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2019. Doubly Robust Joint Learning for Recommendation on Data Missing Not At Random. In International Conference on Machine Learning. PMLR, 6638--6647.
[56]
Xinwei Wu, Hechang Chen, Jiashu Zhao, Li He, Dawei Yin, and Yi Chang. 2021. Unbiased Learning to Rank in Feeds Recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 490--498.
[57]
Longqi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge Belongie, and Deborah Estrin. 2018. Unbiased Offline Recommender Evaluation for Missing-not-at-random Implicit Feedback. In Proceedings of the 12th ACM Conference on Recommender Systems. 279--287.
[58]
Sirui Yao, Yoni Halpern, Nithum Thain, Xuezhi Wang, Kang Lee, Flavien Prost, Ed H. Chi, Jilin Chen, and Alex Beutel. 2021. Measuring Recommender System Effects with Simulated Users. arXiv preprint arXiv:2101.04526 (2021).
[59]
Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal Intervention for Leveraging Popularity Bias in Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 11--20.
[60]
Zhi Zheng, Zhaopeng Qiu, Tong Xu, Xian Wu, Xiangyu Zhao, Enhong Chen, and Hui Xiong. 2022. CBR: Context Bias Aware Recommendation for Debiasing User Modeling and Click Prediction. In Proceedings of the ACM Web Conference 2022. 2268--2276.
[61]
Ziwei Zhu, Yun He, Yin Zhang, and James Caverlee. 2020. Unbiased implicit recommendation and propensity estimation via combinational joint learning. In Proceedings of the 14th ACM Conference on Recommender Systems. 551--556.
[62]
Ziwei Zhu, Yun He, Xing Zhao, and James Caverlee. 2021. Popularity bias in dynamic recommendation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2439--2449.
[63]
Honglei Zhuang, Zhen Qin, Xuanhui Wang, Michael Bendersky, Xinyu Qian, Po Hu, and Dan Chary Chen. 2021. Cross-positional Attention for Debiasing Clicks. In Proceedings of the Web Conference 2021. 788--797.

Cited By

View all
  • (2024)Mitigating Exposure Bias in Online Learning to Rank Recommendation: A Novel Reward Model for Cascading BanditsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679763(1638-1648)Online publication date: 21-Oct-2024

Index Terms

  1. Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender Systems

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval
    July 2024
    3164 pages
    ISBN:9798400704314
    DOI:10.1145/3626772
    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 11 July 2024

    Check for updates

    Author Tags

    1. propensity estimation
    2. recommender systems
    3. unbiased learning

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    SIGIR 2024
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)496
    • Downloads (Last 6 weeks)112
    Reflects downloads up to 04 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Mitigating Exposure Bias in Online Learning to Rank Recommendation: A Novel Reward Model for Cascading BanditsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679763(1638-1648)Online publication date: 21-Oct-2024

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media