skip to main content
research-article

User Response Prediction in Online Advertising

Published: 08 May 2021 Publication History

Abstract

Online advertising, as a vast market, has gained significant attention in various platforms ranging from search engines, third-party websites, social media, and mobile apps. The prosperity of online campaigns is a challenge in online marketing and is usually evaluated by user response through different metrics, such as clicks on advertisement (ad) creatives, subscriptions to products, purchases of items, or explicit user feedback through online surveys. Recent years have witnessed a significant increase in the number of studies using computational approaches, including machine learning methods, for user response prediction. However, existing literature mainly focuses on algorithmic-driven designs to solve specific challenges, and no comprehensive review exists to answer many important questions. What are the parties involved in the online digital advertising eco-systems? What type of data are available for user response prediction? How do we predict user response in a reliable and/or transparent way? In this survey, we provide a comprehensive review of user response prediction in online advertising and related recommender applications. Our essential goal is to provide a thorough understanding of online advertising platforms, stakeholders, data availability, and typical ways of user response prediction. We propose a taxonomy to categorize state-of-the-art user response prediction methods, primarily focusing on the current progress of machine learning methods used in different online platforms. In addition, we also review applications of user response prediction, benchmark datasets, and open source codes in the field.

Supplementary Material

a64-gharibshah-apndx.pdf (gharibshah.zip)
Supplemental movie, appendix, image and software files for, User Response Prediction in Online Advertising

References

[1]
Deepak Agarwal, Rahul Agrawal, Rajiv Khanna, and Nagaraj Kota. 2010. Estimating rates of rare events with multiple hierarchies through scalable log-linear models. In KDD. 213--222.
[2]
Deepak Agarwal, Andrei Zary Broder, Deepayan Chakrabarti, Dejan Diklic, Vanja Josifovski, and Mayssam Sayyadian. 2007. Estimating rates of rare events at multiple resolutions. In KDD. 16--25.
[3]
Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proceedings of the KDD. ACM, 945--955.
[4]
Ethem Alpaydin. 2010. Introduction to Machine Learning (2nd ed.). The MIT Press.
[5]
Kamelia Aryafar, Devin Guillory, and Liangjie Hong. 2017. An ensemble-based approach to click-through rate prediction for promoted listings at Etsy. In Proceedings of the ADKDD. ACM, 6 pages.
[6]
Afroze Ibrahim Baqapuri and Ilya Trofimov. 2014. Using neural networks for click prediction of sponsored search. CoRR abs/1412.6601 (2014).
[7]
D. Barbará, Y. Li, and J. Couto. 2002. COOLCAT: An entropy-based algorithm for categorical clustering. In CIKM. 582--589.
[8]
Eduardo Barbaro, Eoin Martino Grua, Ivano Malavolta, Mirjana Stercevic, Esther Weusthof, and Jeroen van den Hoven. 2020. Modelling and predicting User Engagement in mobile applications. J. Data Sci. 3, 2 (2020), 61--77.
[9]
Nicola Barbieri, Fabrizio Silvestri, and Mounia Lalmas. 2016. Improving post-click user engagement on native ads via survival analysis. In WWW. 761--770.
[10]
Shawn D Baron, Caryn Brouwer, and Amaya Garbayo. 2014. A model for delivering branding value through high-impact digital advertising. J. Advert. Res. 54, 3 (2014), 286--291.
[11]
Sonja Bidmon and Johanna Röttl.2018. Advertising Effects of In-Game-Advertising vs. In-App-Advertising. Springer, 73--86.
[12]
L. Bigon, G. Cassani, C. Greco, L. Lacasa, M. Pavoni, A. Polonioli, and J. Tagliabue. 2019. Prediction is very hard, especially about conversion. Predicting user purchases from clickstream data in fashion e-commerce. CoRR abs/1907.00400 (2019).
[13]
M. Blondel, A. Fujino, N. Ueda, and M. Ishihata. 2016. Higher-order factorization machines. In NIPS. 3359--3367.
[14]
Patrick P. K. Chan, Xian Hu, Lili Zhao, Daniel S. Yeung, Dapeng Liu, and Lei Xiao. 2018. Convolutional neural networks based click-through rate prediction with multiple feature sequences. In IJCAI. 2007--2013.
[15]
Xuchao Zhang Chuxu Zhang Jiashu Zhao Dawei Yin Chao Huang, Xian Wu and Nitesh Chawla. 2019. Online purchase prediction via multi-scale modeling of behavior dynamics. In KDD, 2613--2622.
[16]
Olivier Chapelle. 2014. Modeling delayed feedback in display advertising. In KDD. 1097--1105.
[17]
Olivier Chapelle, Eren Manavoglu, and Rómer Rosales. 2014. Simple and scalable response prediction for display advertising. ACM Trans. Intell. Syst. Technol. 5, 4, Article 61 (2014), 61:1--61:34.
[18]
Gong Chen, Jacob H. Cox, A. Selcuk Uluagac, and John A. Copeland. 2016. In-depth survey of digital advertising technologies. IEEE Commu. Surv. Tutor. 18 (2016), 2124--2148.
[19]
Junxuan Chen, Baigui Sun, Hao Li, Hongtao Lu, and Xian-Sheng Hua. 2016. Deep CTR prediction in display advertising. In Proceedings of the MM. ACM, 811--820.
[20]
Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In SIGIR. 335--344.
[21]
Wenqiang Chen, Lizhang Zhan, Yuanlong Ci, Minghua Yang, Chen Lin, and Dugang Liu. 2019. FLEN: Leveraging field for scalable CTR prediction. CoRR abs/1911.04690 (2019).
[22]
H. Cheng, L. Koc, J. Harmsen, and et al.2016. Wide & deep learning for recommender systems. In DLRS. 7--10.
[23]
Hana Choi, Carl F. Mela, Santiago R. Balseiro, and Adam Leary. 2019. Online display advertising markets: A literature review and future directions. J. Inf. Syst. Res. 31 (2019), 556--575.
[24]
Shu-Chuan Chu. 2011. Viral advertising in social media: Participation in Facebook groups and responses among college-aged users. J. Interact. Advert. 12 (2011), 30--43.
[25]
P. Covington, J. Adams, and E. Sargin. 2016. Deep neural networks for YouTube recommendations. In RecSys. 191--198.
[26]
Brian Dalessandro, Daizhuo Chen, Troy Raeder, Claudia Perlich, Melinda Han Williams, and Foster Provost. 2014. Scalable hands-free transfer learning for online advertising. In KDD. 1573--1582.
[27]
Brian Dalessandro, Rod Hook, Claudia Perlich, and Foster Provost. 2015. Evaluating and optimizing online advertising: Forget the click, but there are good proxies. J. Big Data 3 (2015), 90--102.
[28]
Y. Deng, Y. Shen, and H. Jin. 2017. Disguise adversarial networks for click-through rate prediction. In IJCAI. 1589--1595.
[29]
Ernesto Diaz-Aviles, Lucas Drumond, Lars Schmidt-Thieme, and Wolfgang Nejdl. 2012. Real-time top-n recommendation in social streams. In RecSys. 59--66.
[30]
Daniel M. Dunlavy, Tamara G. Kolda, and Evrim Acar. 2011. Temporal link prediction using matrix and tensor factorizations. ACM Trans. Knowl. Discov. Data 5 (2011), 10:1--10:27.
[31]
N. Ebadi, B. Lwowski, M. Jaloli, and P. Rad. 2019. Implicit life event discovery from call transcripts using temporal input transformation network. IEEE Access 7 (2019), 172178--172189.
[32]
Bora Edizel, Amin Mantrach, and Xiao Bai. 2017. Deep character-level click-through rate prediction for sponsored search. In Proceedings of the SIGIR. ACM, 305--314.
[33]
Muhammad Junaid Effendi and Syed Abbas Ali. 2017. Click through rate prediction for contextual advertisment using linear regression. CoRR abs/1701.08744 (2017).
[34]
F. Maurizio, C. Paolo, and J. Dietmar.2020. Methodological issues in recommender systems research. In IJCAI. 4706--4710.
[35]
Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019. Metapath-guided heterogeneous graph neural network for intent recommendation. ACM Trans. Knowl. Discov. Data (2019), 2478--2486.
[36]
Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, and Keping Yang. 2019. Deep session interest network for click-through rate prediction. In IJCAI. 2301--2307.
[37]
Hongchang Gao, Deguang Kong, Miao Lu, Xiao Bai, and Jian Yang. 2018. Attention convolutional neural network for advertiser-level click-through rate forecasting. In WWW. 1855--1864.
[38]
T. Ge, L. Zhao, G. Zhou, and et al.2018. Image matters: Visually modeling user behaviors using advanced model server. In CIKM, 2087--2095.
[39]
Zhabiz Gharibshah, Xingquan Zhu, Arthur Hainline, and M. Conway. 2020. Deep learning for user interest and response prediction in online display advertising. Data Sci. Eng. 5 (2020), 12--26.
[40]
Djordje Gligorijevic, Jelena Gligorijevic, and A. Flores. 2019. Time-aware prospective modeling of users for online display advertising. CoRR abs/1911.05100 (2019).
[41]
Jelena Gligorijevic, Djordje Gligorijevic, Ivan Stojkovic, Xiao Bai, Amit Goyal, and Zoran Obradovic. 2018. Deeply supervised semantic model for click-through rate prediction in sponsored search. CoRR abs/1803.10739 (2018).
[42]
Thore Graepel, Joaquin Quiñonero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale bayesian click-through rate prediction for sponsored search advertising in Microsoft’s bing search engine. In ICML. 13--20.
[43]
Cheng Guo and Felix Berkhahn. 2016. Entity embeddings of categorical variables. CoRR abs/1604.06737 (2016).
[44]
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A factorization-machine based neural network for CTR prediction. In IJCAI. 1725--1731.
[45]
H. Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, X. He, and Zhenhua Dong. 2018. DeepFM: An end-to-end wide & deep learning framework for CTR prediction. CoRR abs/1804.04950 (2018).
[46]
Rajan T. Gupta and Saibal K. Pal. 2019. Click-through rate estimation using CHAID classification tree model. In Advances in Analytics and Applications. 45--58.
[47]
H. Dustin, S. Stefan, M. Eren, R. Hema, and L. Chirs.2010. Improving ad relevance in sponsored search. In WSDM. 361--370.
[48]
Xinran He, Stuart Bowers, Joaquin Quiñonero Candela, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, and Ralf Herbrich. 2014. Practical lessons from predicting clicks on ads at Facebook. In ADKDD. 1--9.
[49]
Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In SIGIR. 355--364.
[50]
X. He, K. Deng, X. Wang, Y. Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In SIGIR, 639--648.
[51]
X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua. 2017. Neural collaborative filtering. In WWW. 173--182.
[52]
Y. Hu, Y. Koren, and C. Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In ICDM, 263--272.
[53]
Tongwen Huang, Zhiqi Zhang, and Junlin Zhang. 2019. FiBiNET: Combining feature importance and bilinear feature interaction for click-through rate prediction. In RecSys. 169--177.
[54]
Dietmar Jannach, Gabriel de Souza P. Moreira, and Even Oldridge. 2020. Why are deep learning models not consistently winning recommender systems competitions yet? A position paper. In RecSys. 44--49.
[55]
Zilong Jiang, Shu Gao, and Wei Dai. 2016. Research on CTR prediction for contextual advertising based on deep architecture model. Contr. Eng. Appl. Inf. 18 (Mar. 2016), 11--19.
[56]
Zilong Jiang, S. X. Gao, and Mingjiang Li. 2018. An improved advertising CTR prediction approach based on the fuzzy deep neural network. PLoS ONE.
[57]
Yuchin Juan, Damien Lefortier, and Olivier Chapelle. 2017. Field-aware factorization machines in a real-world online advertising system. In Proceedings of the WWW. International World Wide Web Conferences Steering Committee, 680--688.
[58]
Shubhra Karmaker, Parikshit Sondhi, and ChengXiang Zhai. 2017. On application of learning to rank for E-commerce search. In SIGIR.
[59]
K-M Kim, D. Kwak, H. Kwak, Y-J Park, S. Sim, J-H Cho, M. Kim, J. Kwon, Nako Sung, and J-W Ha. 2019. Tripartite heterogeneous graph propagation for large-scale social recommendation. In RecSys. 56--60.
[60]
Michael A. King, Alan S. Abrahams, and Cliff T. Ragsdale. 2015. Ensemble learning methods for pay-per-click campaign management. Expert Syst. Appl. 42 (2015), 4818--4829.
[61]
Y. Koren. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In KDD. 426--434.
[62]
N. Kota and D. Agarwal. 2011. Temporal multi-hierarchy smoothing for estimating rates of rare events. In KDD. 1361--1369.
[63]
S. Krishnan and R. Sitaraman. 2013. Understanding the effectiveness of video ads: A measurement study. In IMC. 149--162.
[64]
S. Ktena, A. Tejani, L. Theis, P. Myana, D. Dilipkumar, F. Huszar, S. Yoo, and W. Shi. 2019. Addressing delayed feedback for continuous training with neural networks in CTR prediction. In Proceedings of the RecSys. ACM, 187--195.
[65]
Ashish Kumar and Jari Salo. 2016. Effects of link placements in email newsletters on their click-through rate. J. Market. Commun. 24, 5 (Mar. 2016), 535--548.
[66]
Rohan Kumar, Mohit Kumar, Neil Shah, and Christos Faloutsos. 2018. Did we get it right? Predicting query performance in e-commerce search. CoRR abs/1808.00239 (2018).
[67]
Mounia Lalmas, Janette Lehmann, Guy Shaked, Fabrizio Silvestri, and Gabriele Tolomei. 2015. Promoting positive post-click experience for in-stream Yahoo Gemini users. In KDD. 1929--1938.
[68]
Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past erformance data. In KDD. 768--776.
[69]
S. Leong, M. Mahdian, and S. Vassilvitskii. 2014. Advertising in a stream. In WWW.
[70]
Cheng Li, Yue Lu, Qiaozhu Mei, Dong Wang, and Sandeep Pandey. 2015. Click-through prediction for advertising in Twitter timeline. In KDD. 1959--1968.
[71]
Feng Li, Zhenrui Chen, Pengjie Wang, Yi Ren, Di Zhang, and Xiaoyu Zhu. 2019. Graph intention network for click-through rate prediction in sponsored search. In SIGIR, 961--964.
[72]
Xiang Li, Chao Wang, Jiwei Tan, Xiaoyi Zeng, Dan Ou, and Bo Zheng. 2020. Adversarial multimodal representation learning for click-through rate prediction. In Proceedings of the Web Conference. ACM, 827--836.
[73]
Zeyu Li, Wei Cheng, Yang Chen, Haifeng Chen, and Wei Wang. 2020. Interpretable click-through rate prediction through hierarchical attention. In WSDM, 313--321.
[74]
Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Fi-GNN: Modeling feature interactions via graph neural networks for CTR prediction. In CIKM. 539--548.
[75]
J. Lian, X. Zhou, F. Zhang, Z. Chen, X. Xie, and G. Sun. 2018. xDeepFM: Combining explicit and implicit feature interactions for recommender systems. In KDD (2018).
[76]
X. Lin, H. Chen, C. Pei, F. Sun, X. Xiao, H. Sun, Y. Zhang, W. Ou, and P. Jiang. 2019. A pareto-efficient algorithm for multiple objective optimization in e-commerce recommendation. In RecSys. 20--28.
[77]
X. Ling, W. Deng, C. Gu, and et al.2017. Model ensemble for click prediction in bing search ads. In WWW. 689--698.
[78]
Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, and Yuzhou Zhang. 2019. Feature generation by convolutional neural network for click-through rate prediction. In WWW. 1119--1129.
[79]
Bin Liu, Niannan Xue, Huifeng Guo, Ruiming Tang, Stefanos Zafeiriou, Xiuqiang He, and Zhenguo Li. 2020. AutoGroup: Automatic feature grouping for modelling explicit high-order feature interactions in CTR prediction.
[80]
Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, X. He, Z. Li, and Y. Yu. 2020. AutoFIS: Automatic feature interaction selection in factorization models for click-through rate prediction. In KDD. 2636--2645.
[81]
Hui Liu, Xingquan Zhu, Kristopher Kalish, and Jeremy Kayne. 2017. ULTR-CTR: Fast page grouping using URL truncation for real-time click through rate estimation. In IEEE IRI.
[82]
Qiang Liu, Shu Wu, and Liang Wang. 2015. Collaborative prediction for multi-entity interaction with hierarchical representation. In CIKM. 613--622.
[83]
Qiang Liu, Feng Yu, Shu Wu, and Liang Wang. 2015. A convolutional click prediction model. In CIKM. 1743--1746.
[84]
Xun Liu, Wei Xue, Lei Xiao, and Bo Zhang. 2017. PBODL: Parallel Bayesian online deep learning for click-through rate prediction in tencent advertising system. CoRR abs/1707.00802 (2017).
[85]
Yozen Liu, Xiaolin Shi, Lucas Pierce, and Xiang Ren. 2019. Characterizing and forecasting user engagement with in-app action graph: A case study of Snapchat. In KDD, 2023--2031.
[86]
Zhe Liu, Xianzhi Wang, Lina Yao, Jake An, Lei Bai, and Ee-Peng Lim. 2020. Face to purchase: Predicting consumer choices with structured facial and behavioral traits embedding. CoRR abs/2007.06842 (2020).
[87]
Amit Livne, Roy Dor, Eyal Mazuz, Tamar Didi, Bracha Shapira, and Lior Rokach. 2020. Iterative boosting deep neural networks for predicting click-through rate. CoRR abs/2007.13087 (2020).
[88]
Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In SIGIR. 1137--1140.
[89]
Miriam Marciel, Rubén Cuevas, Albert Banchs, Roberto González, Stefano Traverso, Mohamed Ahmed, and Arturo Azcorra. 2016. Understanding the detection of view fraud in video content portals. In WWW. 357--368.
[90]
P. Mattson, C. Cheng, C. Coleman, G. Diamos, and et al.2019. MLPerf training benchmark. CoRR abs/1910.01500.
[91]
Stephen McCreery and Dean M. Krugman. 2017. Tablets and TV advertising: Understanding the viewing experience. J. Curr. Issues Res. Advert. 38, 2 (Mar. 2017), 197--211.
[92]
B. McMahan, G. Holt, D. Sculley, and et al.2013. Ad click prediction: A view from the trenches. In KDD. 1222--1230.
[93]
Tao Mei, Xian-Sheng Hua, Linjun Yang, and Shipeng Li. 2007. VideoSense: Towards effective online video advertising. In MM. 1075--1084.
[94]
Wei Meng, Xinyu Xing, Anmol Sheth, Udi Weinsberg, and Wenke Lee. 2014. Your online interests: Pwned! A pollution attack against targeted advertising. In CCS. 129--140.
[95]
Aditya Krishna Menon, Krishna-Prasad Chitrapura, Sachin Garg, Deepak Agarwal, and Nagaraj Kota. 2011. Response prediction using collaborative filtering with hierarchies and side-information. In KDD. 141--149.
[96]
M. Naumov, D. Mudigere, H. Michael Shi, and et. al.2019. Deep learning recommendation model for personalization and recommendation systems. CoRR abs/1906.00091 (2019).
[97]
Chenglei Niu, Guojing Zhong, Y. Liu, Yandong Zhang, Y. Sun, Ailong He, and Zhaoji Chen. 2018. Unstructured semantic model supported deep neural network for click-through rate prediction. CoRR abs/1812.01353 (2018).
[98]
R. Oentaryo, E. Lim, M. Finegold, and et al.2014. Detecting click fraud in online advertising: A data mining approach. J. Mach. Learn. Res. 15 (2014), 99--140.
[99]
R. Oentaryo, E. Lim, J. Low, D. Lo, and M. Finegold. 2014. Predicting response in mobile advertising with hierarchical importance-aware factorization machine. In WSDM. 123--132.
[100]
Wentao Ouyang, Xiuwu Zhang, Li Li, Heng Zou, Xin Xing, Zhaojie Liu, and Yanlong Du. 2019. Deep spatio-temporal neural networks for click-through rate prediction. In KDD (2019), 2078--2086.
[101]
Wentao Ouyang, Xiuwu Zhang, Shukui Ren, Li Li, Zhaojie Liu, and Yanlong Du. 2019. Click-through rate prediction with the user memory network. In Proceedings of the DLP-KDD. ACM, 4 pages.
[102]
Wentao Ouyang, Xiuwu Zhang, Shukui Ren, Chao Qi, Zhaojie Liu, and Yanlong Du. 2019. Representation learning-assisted click-through rate prediction. In IJCAI.
[103]
Feiyang Pan, Shuokai Li, Xiang Ao, Pingzhong Tang, and Qing He. 2019. Warm up cold-start advertisements: Improving CTR predictions via learning to learn ID embeddings. In SIGIR, 695--704.
[104]
Junwei Pan, Yizhi Mao, Alfonso Lobos Ruiz, Yu Sun, and Aaron Flores. 2019. Predicting different types of conversions with multi-task learning in online advertising. In KDD’19 (2019), 2689--2697.
[105]
Jing Pan, Weian Sheng, and Santanu Dey. 2019. Order matters at fanatics recommending sequentially ordered products by LSTM embedded with Word2Vec. CoRR abs/1911.09818 (2019).
[106]
Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018. Field-weighted factorization machines for click-through rate prediction in display advertising. In WWW. 1349--1357.
[107]
Zhen Pan, Enhong Chen, Qi Liu, Tong Xu, Haiping Ma, and Hongjie Lin. 2016. Sparse factorization machines for click-through rate prediction. In Proceedings of the IEEE 16th Intl. Conf. on Data Mining (ICDM’16). 400--409.
[108]
Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou, and Yongfeng Zhang. 2019. Value-aware recommendation based on reinforcement profit maximization. In WWW. 3123--3129.
[109]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In Proceedings of the KDD. ACM, 701--710.
[110]
Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In ACM SIGKDD.
[111]
M. Safaei Pour, A. Mangino, K. Friday, Matthias Rathbun, E. Bou-Harb, F. Iqbal, Kh. Shaban, and A. Erradi. 2019. Data-driven curation, learning and analysis for inferring evolving IoT botnets in the wild. In ARES. Article 6.
[112]
S. Punjabi and P. Bhatt. 2018. Robust factorization machines for user response prediction. In WWW. 669--678.
[113]
P. Qi, X. Zhu, G. Zhou, Y. Zhang, Z. Wang, L. Ren, Y. Fan, and K. Gai. 2020. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the CIKM. ACM, 2685--2692.
[114]
Jiarui Qin, Weinan Zhang, Xin Wu, Jiarui Jin, Yuchen Fang, and Yong Yu. 2020. User behavior retrieval for click-through rate prediction. In Proceedings of the SIGIR. ACM, 2347--2356.
[115]
Ruihong Qiu, Zi Huang, Jingjing Li, and Hongzhi Yin. 2020. Exploiting cross-session information for session-based recommendation with graph neural networks. ACM Transactions on Information Systems 38 (2020).
[116]
Yanru Qu, Ting Bai, Weinan Zhang, Jianyun Nie, and Jian Tang. 2019. An end-to-end neighborhood-based interaction model for knowledge-enhanced recommendation. In Proceedings of the DLP-KDD. ACM, 9 pages.
[117]
Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-based neural networks for user response prediction. In ICDM, 1149--1154.
[118]
Yanru Qu, Bohui Fang, Wei-Nan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, yong Yu, and Xiuqiang He. 2018. Product-based neural networks for user response prediction over multi-field categorical data. ACM Transactions on Information Systems (2018).
[119]
Regelson, Moira, Fain, and Daniel C. 2006. Predicting click-through rate using keyword clusters.
[120]
K. Ren, J. Qin, Y. Fang, W. Zhang, L. Zheng, W. Bian, G. Zhou, J. Xu, Y. Yu, X. Zhu, and K. Gai. 2019. Lifelong sequential modeling with personalized memorization for user response prediction. In SIGIR’19.
[121]
Kan Ren, Weinan Zhang, Yifei Rong, Haifeng Zhang, Yong Yu, and Jun Wang. 2016. User response learning for directly optimizing campaign performance in display advertising. In CIKM. 679--688.
[122]
Steffen Rendle. 2010. Factorization machines. In ICDM, 995--1000.
[123]
S. Rendle, L. Zhang, and Y. Koren. 2019. On the difficulty of evaluating baselines: A study on recommender systems. CoRR abs/1905.01395 (2019).
[124]
Jenna Reps, Uwe Aickelin, Jonathan Garibaldi, and Chris Damski. 2014. Personalising mobile advertising based on users’ installed apps. In ICDM Workshop, 338--345.
[125]
Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: Estimating the click-through rate for new ads. In WWW. 521--530.
[126]
Qiu Ruihong, Yin Hongzhi, Huang Zi, and Tong Chen. 2020. GAG: Global attributed graph neural network for streaming session-based recommendation. In Proceedings of the SIGIR. ACM, 669--678.
[127]
Oliver Rutz, Ashwin Aravindakshan, and Olivier Rubel. 2019. Measuring and forecasting mobile game app engagement. Int. J. Res. Market. 36, 2 (Jun. 2019), 185--199.
[128]
Oliver J. Rutz and Randolph E. Bucklin. 2011. From generic to branded: A model of spillover in paid search advertising. J. Market. Res. (2011), 87--102.
[129]
Rubén Saborido, Foutse Khomh, Giuliano Antoniol, and Yann-Gaël Guéhéneuc. 2017. Comprehension of ads-supported and paid android applications: Are they different? In ICPC, 143--153.
[130]
Ying Shan, T. Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and J. C. Mao. 2016. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In KDD. 255--262.
[131]
Neha Sharma and Nirmal Gaud. 2015. K-modes clustering algorithm for categorical data. Int. J. Comput. Appl. 127 (2015), 1--6.
[132]
Weichen Shen. 2018. Easy-to-use, Modular and Extendible Package of Deep-learning Based CTR Models. Retrieved from https://github.com/shenweichen/DeepCTR.
[133]
Weichen Shen. 2019. (PyTorch) Easy-to-use, Modular and Extendible Package of Deep-learning Based CTR Models. Retrieved from https://github.com/shenweichen/DeepCTR-Torch.
[134]
Shu-Ting Shi, Wenhao Zheng, Jun Tang, Qing-Guo Chen, Yao Hu, Jianke Zhu, and Ming Li. 2020. Deep time-stream framework for click-through rate prediction by tracking interest evolution. AAAI 34, 4 (2020), 5726--5733.
[135]
Enno Shioji and Masayuki Arai. 2017. Neural feature embedding for user response prediction in real-time bidding (RTB). CoRR abs/1702.00855 (2017).
[136]
Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proceedings of the KDD. ACM, 945--955.
[137]
Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. AutoInt: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of the CIKM. ACM, 1161--1170.
[138]
Yuhan Su, Zhongming Jin, Ying Chen, Xinghai Sun, Yaming Yang, Fangzheng Qiao, Fen Xia, and Wei Xu. 2017. Improving click-through rate prediction accuracy in online advertising by transfer learning. In WI. 1018--1025.
[139]
Anh-Phuong Ta. 2015. Factorization machines with follow-the-regularized-leader for CTR prediction in display advertising. In IEEE Big Data (2015), 2889--2891.
[140]
G. S. Thejas, Kianoosh G. Boroojeni, Kshitij Chandna, Isha Bhatia, S. S. Iyengar, and N. R. Sunitha. 2019. Deep Learning-based Model to Fight Against Ad Click Fraud. In ACM SE. 176--181.
[141]
T. Tian, J. Zhu, F. Xia, X. Zhuang, and T. Zhang. 2015. Crowd fraud detection in internet advertising. In WWW. 1100--1110.
[142]
Gabriele Tolomei, Mounia Lalmas, Ayman Farahat, and Andrew Haines. 2018. You must have clicked on this ad by mistake! Data-driven identification of accidental clicks on mobile ads with applications to advertiser cost discounting and click-through rate prediction. Data Sci. Analyt. 7 (2018), 53--66.
[143]
Sergio Duarte Torres, Ingmar Weber, and Djoerd Hiemstra. 2014. Analysis of search and browsing behavior of young users on the web. ACM Trans. Web 8 (2014), 7:1--7:54.
[144]
Vinh Truong, Mathews Nkhoma, and Wanni Pansuwong. 2019. An integrated effectiveness framework of mobile in-app advertising. Australas. J. Inf. Syst. 23 (2019).
[145]
Flavian Vasile, Damien Lefortier, and Olivier Chapelle. 2017. Cost-sensitive learning for utility optimization in online advertising auctions. In Proceedings of the ADKDD. 1--6.
[146]
Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. RippleNet: Propagating user preferences on the knowledge graph for recommender systems. In CIKM’18, 417--426.
[147]
Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. DKN: Deep knowledge-aware network for news recommendation. In Proceedings of the WWW. International World Wide Web Conferences Steering Committee, 1835--1844.
[148]
H. Wang, F. Zhang, M. Zhang, J. Leskovec, M. Zhao, W. Li, and Z. Wang. 2019. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems. In KDD (2019), 968--977.
[149]
Hongwei Wang, Fuzheng Zhang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2019. Multi-task feature learning for knowledge graph enhanced recommendation. In WWW. 2000--2010.
[150]
Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale commodity embedding for E-commerce recommendation in Alibaba. KDD, 839--848.
[151]
Jun Wang, Weinan Zhang, and Shuai Yuan. 2016. Display advertising with real-time bidding (RTB) and behavioural targeting. Found. Trends Inf. Retr. 11 (2016), 297--435.
[152]
Qianqian Wang, Fang’ai Liu, Shuning Xing, and Xiaohui Zhao. 2018. A new approach for advertising CTR prediction based on deep neural network via attention mechanism. Comp. Math. Methods Med. 2018 (2018), 1--11.
[153]
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29 (2017), 2724--2743.
[154]
Ruoxi Wang, Bin Fu, Gang Fu, et al. 2017. Deep & cross network for ad click predictions. In Proceedings of the ADKDD. ACM, 7 pages.
[155]
Ruoxi Wang, Rakesh Shivanna, D. Cheng, S. Jain, D. Lin, L. Hong, and Ed Huai hsin Chi. 2020. DCN V2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. arXiv arXiv:2008.13535.
[156]
X. Wang, X. He, M. Wang, F. Feng, and T. Chua. 2019. Neural graph collaborative filtering. In Proceedings of the SIGIR. ACM, 165--174.
[157]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Peng Cui, P. Yu, and Yanfang Ye. 2019. Heterogeneous graph attention network. In Proceedings of the World Wide Web Conference. ACM, 2022--2032.
[158]
X. Wang, W. Li, Y. Cui, R. Zhang, and J. Mao. 2011. Click-through rate estimation for rare events in online advertising.
[159]
Hong Wen, Jing Zhang, Quan Lin, Keping Yang, and Pipei Huang. 2019. Multi-level deep cascade trees for conversion rate prediction. AAAI 33 (2019), 338--345.
[160]
Hong Wen, Jing Zhang, Yuan Wang, Wentian Bao, Quan Lin, and Keping Yang. 2019. Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction. In Proceedings of the SIGIR. ACM, 2377--2386.
[161]
Xian Wu, Baoxu Shi, Yuxiao Dong, Chao Huang, and Nitesh V. Chawla. 2019. Neural tensor factorization for temporal interaction learning. In WSDM. 537--545.
[162]
Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In Proceedings of the IJCAI. AAAI Press, 3119--3125.
[163]
Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang, Changhua Pei, Fei Sun, Jian Wu, Hanxiao Sun, and Wenwu Ou. 2020. Privileged features distillation at Taobao recommendations. In Proceedings of the KDD. ACM, 2590--2598.
[164]
C. Xu and M. Wu. 2020. Learning feature interactions with lorentzian factorization machine. In AAAI. 6470--6477.
[165]
Hongxia Yang, Quan Lu, Angus Xianen Qiu, and Chun Han. 2016. Large scale CVR prediction through dynamic transfer learning of global and local features. In Proceedings of Machine Learning Research, Vol. 53. 103--119.
[166]
Xiao Yang, Tao Deng, Weihan Tan, Xutian Tao, Junwei Zhang, Shouke Qin, and Zongyao Ding. 2019. Learning compositional, visual and relational representations for CTR prediction in sponsored search. In CIKM. 2851--2859.
[167]
Y. Yang, X. Guan, and J. You. 2002. CLOPE: A fast and effective clustering algorithm for transactional data. In KDD.
[168]
Yi Yang, Baile Xu, Furao Shen, and Jian Zhao. 2019. Operation-aware neural networks for user response prediction. Neural Netw. 121 (2019), 161--168.
[169]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. CoRR abs/1806.01973 (2018).
[170]
Xu Yong, Chen Jiahui, Huang Chao, Zhang Bo, Xing Hao, Dai Peng, and Bo Liefeng. 2020. Joint modeling of local and global behavior dynamics for session-based recommendation*. In ECAI, 545--552.
[171]
Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang. 2020. Parameter-efficient transfer from sequential behaviors for user modeling and recommendation. In SIGIR. 1469--1478.
[172]
Yuan Yuan, Xiaojing Dong, Chen Dong, Yiwen Sun, Zhenyu Yan, and Abhishek Pani. 2018. Dynamic hierarchical empirical Bayes: A predictive model applied to online advertising. CoRR abs/1809.02213 (2018).
[173]
Y. Yuan, F. Wang, J. Li, and R. Qin. 2014. A survey on real time bidding advertising. In IEEE SOLI. 418--423.
[174]
C. Zhang, D. Song, C. Huang, A. Swami, and N. Chawla. 2019. Heterogeneous graph neural network. In KDD. 793--803.
[175]
Chuxu Zhang, A. Swami, and Nitesh V. Chawla. 2019. SHNE: Representation learning for semantic-associated heterogeneous networks. In WSDM (2019).
[176]
J. Zhang, T. Huang, and Z. Zhang. 2019. FAT-DeepFFM: Field attentive deep field-aware factorization machine. In ICDM.
[177]
Li Zhang, Weichen Shen, Shijian Li, and Gang Pan. 2019. Field-aware neural factorization machine for click-through rate prediction. IEEE Access 7 (2019), 75032--75040.
[178]
Weinan Zhang, Lingxi Chen, and Jun Wang. 2016. Implicit look-alike modelling in display ads - Transfer collaborative filtering to CTR estimation. CoRR abs/1601.02377 (2016).
[179]
Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data - - A case study on user response prediction. In ECIR.
[180]
Weinan Zhang, Tianxiong Zhou, Jun Wang, and Jian Xu. 2016. Bid-aware gradient descent for unbiased learning with censored data in display advertising. In KDD. 665--674.
[181]
Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, and Tie-Yan Liu. 2014. Sequential click prediction for sponsored search with recurrent neural networks. In AAAI. 1369--1375.
[182]
Yang Zhang, Fuli Feng, Chenxu Wang, Xiangnan He, Meng Wang, Yan Li, and Yongdong Zhang. 2020. How to retrain recommender system? A sequential meta-learning method. In SIGIR. 1479--1488.
[183]
Y. Zhang, P. Zhao, Y. Guan, L. Chen, K. Bian, L. Song, B. Cui, and X. Li. 2020. Preference-aware mask for session-based recommendation with bidirectional transformer. In ICASSP. 3412--3416.
[184]
Weijie Zhao, Deping Xie, Ronglai Jia, Yulei Qian, Ruiquan Ding, Mingming Sun, and Ping Li. 2020. Distributed hierarchical GPU parameter server for massive scale deep learning ads systems. CoRR abs/2003.05622 (2020).
[185]
Xiangyu Zhao, Xudong Zheng, Xiwang Yang, Xiaobing Liu, and Jiliang Tang. 2020. Jointly learning to recommend and advertise. In Proceedings of the KDD. ACM, 3319--3327.
[186]
Yifei Zhao, Yu-Hang Zhou, Mingdong Ou, Huan Xu, and Nan Li. 2020. Maximizing cumulative user engagement in sequential recommendation: An online optimization perspective. In KDD. 2784--2792.
[187]
Hua Zheng, Dong Wang, Qi Zhang, Hang Li, and Tinghao Yang. 2010. Do clicks measure recommendation relevancy? An empirical user study. In RecSys. 249--252.
[188]
Guorui Zhou, Ying Fan, Runpeng Cui, Weijie Bian, Xiaoqiang Zhu, and Kun Gai. 2018. Rocket launching: A universal and efficient framework for training well-performing light net. AAAI 32, 1 (2018).
[189]
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. AAAI 33 (2019), 5941--5948.
[190]
Guorui Zhou, Chengru Song, Xiaoqiang Zhu, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD (2018), 1059--1068.
[191]
Guorui Zhou, Kailun Wu, Weijie Bian, Zhao Yang, Xiaoqiang Zhu, and Kun Gai. 2019. Res-embedding for deep learning based click-through rate prediction modeling. In Proceedings of the DLP-KDD. ACM, 9 pages.
[192]
K. Zhou, M. Redi, A. Haines, and et al.2016. Predicting pre-click quality for native advertisements. In WWW. 299--310.
[193]
Wen-Yuan Zhu, Chun-Hao Wang, Wen-Yueh Shih, W. Peng, and J. Huang. 2017. SEM: A softmax-based ensemble model for CTR estimation in real-time bidding advertising. In IEEE BigComp.5--12.
[194]
Xingquan Zhu and Ian Davidson. 2007. Knowledge Discovery and Data Mining: Challenges and Realities. IGI Global.
[195]
X. Zhu, H. Tao, Z. Wu, J. Cao, K. Kalish, and J. Kayne. 2017. Fraud Prevention in Online Digital Advertising. Springer.
[196]
B. Zoph and Q. Le. 2016. Neural architecture search with reinforcement learning. CoRR abs/1611.01578 (2016).
[197]
Zhabiz Gharibshah and Xingquan Zhu. 2020. TriNE: Network representation learning for tripartite heterogeneous networks. In IEEE International Conference on Knowledge Graph (ICKG’20). IEEE, 497--504.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys
ACM Computing Surveys  Volume 54, Issue 3
April 2022
836 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/3461619
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021
Accepted: 01 January 2021
Revised: 01 November 2020
Received: 01 April 2020
Published in CSUR Volume 54, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Click
  2. bounce rate
  3. conversion
  4. convolutional neural network
  5. data management platform
  6. deep learning
  7. demand side platform
  8. dwell time
  9. factorization machines
  10. graph neural network
  11. impression
  12. knowledge graph
  13. landing page
  14. recurrent neural network
  15. supplier side platform
  16. user engagement

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • National Science Foundation (CNS)
  • National Science Foundation (IIS)

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)244
  • Downloads (Last 6 weeks)30
Reflects downloads up to 25 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media