research-article

Popularity Prediction of Social Media based on Multi-Modal Feature Mining

Authors:

Chih-Chung Hsu,

Zhong-Xuan Zhang,

Shao-Min WuAuthors Info & Claims

MM '19: Proceedings of the 27th ACM International Conference on Multimedia

Pages 2687 - 2691

https://doi.org/10.1145/3343031.3356064

Published: 15 October 2019 Publication History

Abstract

Popularity prediction of social media becomes a more attractive issue in recent years. It consists of multi-type data sources such as image, meta-data, and text information. In order to effectively predict the popularity of a specified post in the social network, fusing multi-feature from heterogeneous data is required. In this paper, a popularity prediction framework for social media based on multi-modal feature mining is presented. First, we discover image semantic features by extracting their image descriptions generated by image captioning. Second, an effective text-based feature engineering is used to construct an effective word-to-vector model. The trained word-to-vector model is used to encode the text information and the semantic image features. Finally, an ensemble regression approach is proposed to aggregate these encoded features and learn the final regressor. Extensive experiments show that the proposed method significantly outperforms other state-of-the-art regression models. We also show that the multi-modal approach could effectively improve the performance in the social media prediction challenge.

References

[1]

Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 785--794.

Digital Library

[2]

Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural Architecture Search: A Survey. Journal of Machine Learning Research, Vol. 20, 55 (2019), 1--21.

[3]

Shintami Chusnul Hidayati, Yi-Ling Chen, Chao-Lung Yang, and Kai-Lung Hua. 2017. Popularity Meter: An Influence- and Aesthetics-aware Social Media Popularity Predictor. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, New York, NY, USA, 1918--1923. https://doi.org/10.1145/3123266.3127903

Digital Library

[4]

Chih-Chung Hsu, Chia-Yen Lee, Ting-Xuan Liao, Jun-Yi Lee, Tsai-Yne Hou, Ying-Chu Kuo, Jing-Wen Lin, Ching-Yi Hsueh, Zhong-Xuan Zhang, and Hsiang-Chin Chien. 2018. An iterative refinement approach for social media headline prediction. In Proceedings of the ACM International Conference on Multimedia. ACM, 2008--2012.

Digital Library

[5]

Chih-Chung Hsu, Ying-Chin Lee, Ping-En Lu, Shian-Shin Lu, Hsiao-Ting Lai, Chihg-Chu Huang, Chun Wang, Yang-Jiun Lin, and Weng-Tai Su. 2017. Social media prediction based on residual learning and random forest. In Proceedings of the ACM International Conference on Multimedia. ACM, 1865--1870.

Digital Library

[6]

Ian Jolliffe. 2011. Principal component analysis. Springer.

[7]

Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems. 3146--3154.

Digital Library

[8]

Liuwu Li, Sihong Huang, Ziliang He, and Wenyin Liu. 2018. An Effective Text-based Characterization Combined with Numerical Features for Social Media Headline Prediction. In Proceedings of the ACM International Conference on Multimedia. ACM, 2003--2007.

Digital Library

[9]

Liuwu Li, Runwei Situ, Junyan Gao, Zhenguo Yang, and Wenyin Liu. 2017. A Hybrid Model Combining Convolutional Neural Network with XGBoost for Predicting Social Media Popularity. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, New York, NY, USA, 1912--1917. https://doi.org/10.1145/3123266.3127902

Digital Library

[10]

Andy Liaw, Matthew Wiener, et almbox. 2002. Classification and regression by randomForest. R News, Vol. 2, 3 (2002), 18--22.

[11]

Jinna Lv, Wu Liu, Meng Zhang, He Gong, Bin Wu, and Huadong Ma. 2017. Multi-feature Fusion for Predicting Social Media Popularity. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, New York, NY, USA, 1883--1888. https://doi.org/10.1145/3123266.3127897

Digital Library

[12]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).

[13]

Wen Wang and Wei Zhang. 2017. Combining Multiple Features for Image Popularity Prediction in Social Media. In Proceedings of the 25th ACM International Conference on Multimedia (MM '17). ACM, New York, NY, USA, 1901--1905. https://doi.org/10.1145/3123266.3127900

Digital Library

[14]

Bo Wu, Wen-Huang Cheng, Yongdong Zhang, and Tao Mei. 2016a. Time Matters: Multi-scale Temporalization of Social Media Popularity. In Proceedings of the ACM International Conference on Multimedia .

Digital Library

[15]

Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Huang Qiushi, Li Jintao, and Tao Mei. 2017. Sequential Prediction of Social Media Popularity with Deep Temporal Context Networks. In Proceedings of the International Joint Conference on Artificial Intelligence .

[16]

Bo Wu, Tao Mei, Wen-Huang Cheng, and Yongdong Zhang. 2016b. Unfolding Temporal Dynamics: Predicting Social Media Popularity Using Multi-scale Temporal Decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence .

[17]

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning. 2048--2057.

Digital Library

[18]

Sheng Yue, Paul Pilon, and George Cavadias. 2002. Power of the Mann--Kendall and Spearman's rho tests for detecting monotonic trends in hydrological series. Journal of hydrology, Vol. 259, 1--4 (2002), 254--271.

Cited By

Hsu CLee CLin YChou YJian CTsai CCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Revisiting Vision-Language Features Adaptation and Inconsistency for Social Media Popularity PredictionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689000(11464-11469)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3689000
Hu WChen WYuan WWang YCai SZhang YCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Dual-Stream Pre-Training Transformer to Enhance Multimodal Learning for Social Media PredictionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3688998(11450-11456)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3688998
Lin YLee ACai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)MMF: Winning Solution to Social Media Popularity Prediction Challenge 2024Proceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3688997(11445-11449)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3688997
Show More Cited By

Index Terms

Popularity Prediction of Social Media based on Multi-Modal Feature Mining
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Social media
2. Information systems
  1. Information retrieval
    1. Specialized information retrieval
      1. Multimedia and multimodal retrieval

Recommendations

Social Media Popularity Prediction: A Multiple Feature Fusion Approach with Deep Neural Networks
MM '19: Proceedings of the 27th ACM International Conference on Multimedia

Social media popularity prediction (SMPD) aims to predict the popularity of the post shared on online social media platforms. This task is crucial for content providers and consumers in a wide range of real-world applications, including multimedia ...
A Feature Generalization Framework for Social Media Popularity Prediction
MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Social media is an indispensable part in modern life and social media popularity prediction can be applied to many aspects of sociality. In this paper, we propose a novel combined framework for social media popularity prediction, which accomplishes ...
Mining social media with social theories: a survey

The increasing popularity of social media encourages more and more users to participate in various online activities and produces data in an unprecedented rate. Social media data is big, linked, noisy, highly unstructured and in- complete, and differs ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '19: Proceedings of the 27th ACM International Conference on Multimedia

October 2019

2794 pages

ISBN:9781450368896

DOI:10.1145/3343031

General Chairs:
Laurent Amsaleg
CNRS-IRISA, France
,
Benoit Huet
EURECOM, France
,
Martha Larson
Radboud University and TU Delft (Netherlands)
,
Program Chairs:
Guillaume Gravier
CNRS-IRISA, France
,
Hayley Hung
Delft University of Technology Netherlands
,
Chong-Wah Ngo
City University of Hong Kong Hong Kong
,
Wei Tsang Ooi
National University of Singapore Singapore

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Ministry of Science and Technology of Taiwan

Conference

MM '19

Sponsor:

SIGMM

MM '19: The 27th ACM International Conference on Multimedia

October 21 - 25, 2019

Nice, France

Acceptance Rates

MM '19 Paper Acceptance Rate 252 of 936 submissions, 27%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
517
Total Downloads

Downloads (Last 12 months)76
Downloads (Last 6 weeks)10

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hsu CLee CLin YChou YJian CTsai CCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Revisiting Vision-Language Features Adaptation and Inconsistency for Social Media Popularity PredictionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689000(11464-11469)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3689000
Hu WChen WYuan WWang YCai SZhang YCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Dual-Stream Pre-Training Transformer to Enhance Multimodal Learning for Social Media PredictionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3688998(11450-11456)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3688998
Lin YLee ACai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)MMF: Winning Solution to Social Media Popularity Prediction Challenge 2024Proceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3688997(11445-11449)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3688997
Wu BLiu PHuang QZeng ZWang JLiu BLuo JCheng WCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)SMP Challenge Summary: Social Media Prediction ChallengeProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3688996(11442-11444)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3688996
Jeong DSon HChoi YKim K(2024)Enhancing social media post popularity prediction with visual contentJournal of the Korean Statistical Society10.1007/s42952-024-00270-753:3(844-882)Online publication date: 21-May-2024
https://doi.org/10.1007/s42952-024-00270-7
Ding SHou XLiu YZhu WFang DFan YLi KHuang LZhou F(2023)DanceTrend: An Integration Framework of Video-Based Body Action Recognition and Color Space Features for Dance Popularity PredictionElectronics10.3390/electronics1222469612:22(4696)Online publication date: 18-Nov-2023
https://doi.org/10.3390/electronics12224696
Wu BLiu PCheng WLiu BZeng ZWang JHuang QLuo JEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)SMP Challenge: An Overview and Analysis of Social Media Prediction ChallengeProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3613853(9651-9655)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3613853
Chen XChen WHuang CZhang ZDuan LZhang YEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Double-Fine-Tuning Multi-Objective Vision-and-Language Transformer for Social Media Popularity PredictionProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612845(9462-9466)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612845
Hsu CLee CHou XTsai CEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Gradient Boost Tree Network based on Extensive Feature Analysis for Popularity Prediction of Social PostsProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612843(9451-9455)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612843
Ji LPark CRao ZChen QEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Neural Image Popularity Assessment with Retrieval-augmented TransformerProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611918(2427-2436)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611918
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents