research-article

Nowcasting Events from the Social Web with Statistical Learning

Authors:

Vasileios Lampos,

Nello CristianiniAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 3, Issue 4

Article No.: 72, Pages 1 - 22

https://doi.org/10.1145/2337542.2337557

Published: 01 September 2012 Publication History

Abstract

We present a general methodology for inferring the occurrence and magnitude of an event or phenomenon by exploring the rich amount of unstructured textual information on the social part of the Web. Having geo-tagged user posts on the microblogging service of Twitter as our input data, we investigate two case studies. The first consists of a benchmark problem, where actual levels of rainfall in a given location and time are inferred from the content of tweets. The second one is a real-life task, where we infer regional Influenza-like Illness rates in the effort of detecting timely an emerging epidemic disease. Our analysis builds on a statistical learning framework, which performs sparse learning via the bootstrapped version of LASSO to select a consistent subset of textual features from a large amount of candidates. In both case studies, selected features indicate close semantic correlation with the target topics and inference, conducted by regression, has a significant performance, especially given the short length --approximately one year-- of Twitter’s data time series.

References

[1]

Asur, S. and Huberman, B. A. 2010. Predicting the future with social media. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. IEEE, 492--499.

Digital Library

[2]

Bach, F. R. 2008. Bolasso: Model consistent Lasso estimation through the bootstrap. In Proceedings of the 25th International Conference on Machine Learning. 33--40.

Digital Library

[3]

Bartlett, P. L., Mendelson, S., and Neeman, J. 2009. l1-regularized linear regression: Persistence and oracle inequalities. Tech. rep., UC-Berkeley.

[4]

Bollen, J., Mao, H., and Zeng, X. 2011. Twitter mood predicts the stock market. J. Comput. Sci.

[5]

Breiman, L. 1996. Bagging predictors. Mach. Learn. 24, 2, 123--140.

Digital Library

[6]

Corley, C. D., Mikler, A. R., Singh, K. P., and Cook, D. J. 2009. Monitoring influenza trends through mining social media. In Proceedings of the International Conference on Bioinformatics and Computational Biology. 340--346.

[7]

Culotta, A. 2010. Towards detecting influenza epidemics by analyzing Twitter messages. In Proceedings of the KDD Workshop on Social Media Analytics.

Digital Library

[8]

Efron, B. 1979. Bootstrap methods: Another look at the jackknife. Ann. Statist. 7, 1, 1--26.

[9]

Efron, B. and Tibshirani, R. J. 1993. An Introduction to the Bootstrap. Chapman & Hall.

[10]

Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. 2004. Least angle regression. Ann. Statist. 32, 2, 407--451.

[11]

Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., and Brilliant, L. 2008. Detecting influenza epidemics using search engine query data. Nature 457, 7232, 1012--1014.

[12]

Guyon, I. and Elisseeff, A. 2003. An introduction to variable and feature selection. J. Mach. Learn. Resear. 3, 7--8, 1157--1182.

Digital Library

[13]

Jenkins, G. J., Perry, M. C., and Prior, M. J. 2008. The Climate of the United Kingdom and Recent Trends. Met Office, Hadley Centre, Exeter, UK.

[14]

Lampos, V. and Cristianini, N. 2010. Tracking the flu pandemic by monitoring the Social Web. In Proceedings of the 2nd IAPR Workshop on Cognitive Information Processing. IEEE Press, 411--416.

[15]

Lampos, V., De Bie, T., and Cristianini, N. 2010. Flu detector---Tracking epidemics on Twitter. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases. Springer, 599--602.

Digital Library

[16]

Lv, J. and Fan, Y. 2009. A unified approach to model selection and sparse recovery using regularized least squares. Ann. Statist. 37, 6A, 3498--3528.

[17]

Manning, C. D., Raghavan, P., and Schütze, H. 2008. Introduction to Information Retrieval. Cambridge University Press.

Digital Library

[18]

Pang, B. and Lee, L. 2008. Opinion mining and sentiment analysis. Found. Trends Inf. Retriev. 2, 1--2, 1--135.

Digital Library

[19]

Polgreen, P. M., Chen, Y., Pennock, D. M., Nelson, F. D., and Weinstein, R. A. 2008. Using internet searches for influenza surveillance. Clinical Infectious Diseases 47, 11, 1443--1448.

[20]

Porter, M. F. 1980. An algorithm for suffix stripping. Program 14, 3, 130--137.

[21]

Sakaki, T., Okazaki, M., and Matsuo, Y. 2010. Earthquake shakes Twitter users: Real-time event detection by social sensors. In Proceedings of the 19th International Conference on World Wide Web. 851--860.

Digital Library

[22]

Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. J. Royal Statist. Soc. Series B (Methodological) 58, 1, 267--288.

[23]

Tumasjan, A., Sprenger, T. O., Sandner, P. G., and Welpe, I. M. 2010. Predicting elections with Twitter: What 140 characters reveal about political sentiment. In Proceedings of the International AAAI Conference on Weblogs and Social Media. 178--185.

[24]

Zhao, P. and Yu, B. 2006. On model selection consistency of Lasso. J. Mach. Learn. Resear. 7, 11, 2541--2563.

Digital Library

Cited By

Kovalchuk MFilatova AKorneev AKoreneva MNasonov DVoskresenskii ABoukhanovsky A(2024)SemConvTree: Semantic Convolutional Quadtrees for Multi-Scale Event Detection in Smart CitySmart Cities10.3390/smartcities70501077:5(2763-2780)Online publication date: 28-Sep-2024
https://doi.org/10.3390/smartcities7050107
Gautam ARaza Z(2024)Disease outbreak prediction using natural language processing: a reviewKnowledge and Information Systems10.1007/s10115-024-02192-666:11(6561-6595)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10115-024-02192-6
Bhunia GShit PBrahma S(2023)GI Science application for groundwater resources management and decision supportCase Studies in Geospatial Applications to Groundwater Resources10.1016/B978-0-323-99963-2.00014-6(27-38)Online publication date: 2023
https://doi.org/10.1016/B978-0-323-99963-2.00014-6
Show More Cited By

Index Terms

Nowcasting Events from the Social Web with Statistical Learning

Recommendations

Supporting temporal analytics for health-related events in microblogs
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge management

Microblogging services, such as Twitter, are gaining interests as a means of sharing information in social networks. Numerous works have shown the potential of using Twitter posts (or tweets) in order to infer the existence and magnitude of real-world ...
Identification of live news events using Twitter
LBSN '11: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Location-Based Social Networks

Twitter presents a source of information that cannot easily be obtained anywhere else. However, though many posts on Twitter reveal up-to-the-minute information about events in the world or interesting sentiments, far more posts are of no interest to ...
Feature Extraction and Analysis for Identifying Disruptive Events from Social Media
ASONAM '15: Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015

Disruptive event identification is a concept that is crucial to ensuring public safety regarding large-scale events. Recent work on detecting events from social media shows that although these platforms are used for social purposes, they have been ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 3, Issue 4

September 2012

410 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/2337542

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2012

Accepted: 01 September 2011

Revised: 01 August 2011

Received: 01 April 2011

Published in TIST Volume 3, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Engineering and Physical Sciences Research Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

116
Total Citations
View Citations
1,144
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)4

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kovalchuk MFilatova AKorneev AKoreneva MNasonov DVoskresenskii ABoukhanovsky A(2024)SemConvTree: Semantic Convolutional Quadtrees for Multi-Scale Event Detection in Smart CitySmart Cities10.3390/smartcities70501077:5(2763-2780)Online publication date: 28-Sep-2024
https://doi.org/10.3390/smartcities7050107
Gautam ARaza Z(2024)Disease outbreak prediction using natural language processing: a reviewKnowledge and Information Systems10.1007/s10115-024-02192-666:11(6561-6595)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10115-024-02192-6
Bhunia GShit PBrahma S(2023)GI Science application for groundwater resources management and decision supportCase Studies in Geospatial Applications to Groundwater Resources10.1016/B978-0-323-99963-2.00014-6(27-38)Online publication date: 2023
https://doi.org/10.1016/B978-0-323-99963-2.00014-6
Binti Abdul Ghani NHamid SAhmad MSaadi YJhanjhi NA. Alzain MMasud M(2022)Tracking Dengue on Twitter Using Hybrid Filtration-Polarity and Apache FlumeComputer Systems Science and Engineering10.32604/csse.2022.01846740:3(913-926)Online publication date: 2022
https://doi.org/10.32604/csse.2022.018467
Cataldi MDi Caro LSchifanella C(2022)SIDEWAYS-2022 @ HT-2022: 7th International Workshop on Social Media World SensorsProceedings of the 7th International Workshop on Social Media World Sensors10.1145/3544795.3544844(1-4)Online publication date: 28-Jun-2022
https://dl.acm.org/doi/10.1145/3544795.3544844
Di Caro LSchifanella CCataldi MBellogín ABoratto LCena F(2022)SIDEWAYS-2022 @ HT-2022: 7th International Workshop on Social Media World SensorsProceedings of the 33rd ACM Conference on Hypertext and Social Media10.1145/3511095.3532573(265-268)Online publication date: 28-Jun-2022
https://dl.acm.org/doi/10.1145/3511095.3532573
Wiegmann MKersten JSenaratne HPotthast MKlan FStein B(2021)Opportunities and risks of disaster data from social media: a systematic review of incident informationNatural Hazards and Earth System Sciences10.5194/nhess-21-1431-202121:5(1431-1444)Online publication date: 6-May-2021
https://doi.org/10.5194/nhess-21-1431-2021
Bruneau PBrangbour EMarchand-Maillet SHostache RChini MPelich RMatgen PTamisier T(2021)Measuring the Impact of Natural Hazards with Citizen Science: The Case of Flooded Area Estimation Using TwitterRemote Sensing10.3390/rs1306115313:6(1153)Online publication date: 18-Mar-2021
https://doi.org/10.3390/rs13061153
Li LNovillo-Ortiz DAzzopardi-Muscat NKostkova P(2021)Digital Data Sources and Their Impact on People's Health: A Systematic Review of Systematic ReviewsFrontiers in Public Health10.3389/fpubh.2021.6452609Online publication date: 5-May-2021
https://doi.org/10.3389/fpubh.2021.645260
Kostkova PSaigí-Rubió FEguia HBorbolla DVerschuuren MHamilton CAzzopardi-Muscat NNovillo-Ortiz D(2021)Data and Digital Solutions to Support Surveillance Strategies in the Context of the COVID-19 PandemicFrontiers in Digital Health10.3389/fdgth.2021.7079023Online publication date: 6-Aug-2021
https://doi.org/10.3389/fdgth.2021.707902
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents