research-article

Towards Ordinal Suicide Ideation Detection on Social Media

Authors:

Rajiv Ratn ShahAuthors Info & Claims

WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Pages 22 - 30

https://doi.org/10.1145/3437963.3441805

Published: 08 March 2021 Publication History

Abstract

The rising ubiquity of social media presents a platform for individuals to express suicide ideation, instead of traditional, formal clinical settings. While neural methods for assessing suicide risk on social media have shown promise, a crippling limitation of existing solutions is that they ignore the inherent ordinal nature across fine-grain levels of suicide risk. To this end, we reformulate suicide risk assessment as an Ordinal Regression problem, over the Columbia-Suicide Severity Scale. We propose SISMO, a hierarchical attention model optimized to factor in the graded nature of increasing suicide risk levels, through soft probability distribution since not all wrong risk-levels are equally wrong. We establish the face value of SISMO for preliminary suicide risk assessment on real-world Reddit data annotated by clinical experts. We conclude by discussing the empirical, practical, and ethical considerations pertaining to SISMO in a larger picture, as a human-in-the-loop framework

References

[1]

Payam Amini, Hasan Ahmadinia, Jalal Poorolajal, and Mohammad Moqaddasi Amiri. 2016. Evaluating the high risk groups for suicide: a comparison of logistic regression, support vector machine, decision tree and artificial neural network. Iranian journal of public health, Vol. 45, 9 (2016), 1179.

[2]

Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The Long-Document Transformer. arXiv:2004.05150 (2020).

[3]

Ralf Bender and Ulrich Grouven. 1997. Ordinal logistic regression in medical research. Journal of the Royal College of physicians of London, Vol. 31, 5 (1997), 546.

[4]

Adrian Benton, Glen Coppersmith, and Mark Dredze. 2017. Ethical research protocols for social media health research. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing. 94--102.

[5]

Chloe Berryman, Christopher J. Ferguson, and Charles Negy. 2017. Social Media Use and Mental Health among Young Adults. Psychiatric Quarterly, Vol. 89, 2 (Nov. 2017), 307--314. https://doi.org/10.1007/s11126-017--9535--6

[6]

Amy Bruckman. 2002. Studying the amateur artist: A perspective on disguising data collected in human subjects research on the Internet. Ethics and Information Technology, Vol. 4, 3 (2002), 217--231.

Digital Library

[7]

Craig J Bryan and M David Rudd. 2006. Advances in the assessment of suicide risk. Journal of clinical psychology, Vol. 62, 2 (2006), 185--200.

[8]

Lei Cao, Huijun Zhang, Ling Feng, Zihan Wei, Xin Wang, Ningyun Li, and Xiaohao He. 2019. Latent Suicide Risk Detection on Microblog via Suicide-Oriented Word Embeddings and Layered Attention. In Proceedings of EMNLP-IJCNLP 2019 .

[9]

Stevie Chancellor, Eric PS Baumer, and Munmun De Choudhury. 2019 a. Who is the" Human" in Human-Centered Machine Learning: The Case of Predicting Mental Health from Social Media. Proceedings of the ACM on HCI (2019).

Digital Library

[10]

Stevie Chancellor, Michael L Birnbaum, Eric D Caine, Vincent MB Silenzio, and Munmun De Choudhury. 2019 b. A taxonomy of ethical tensions in inferring mental health states from social media. In In Proceedings of FAT*. 79--88.

Digital Library

[11]

Stevie Chancellor and Munmun De Choudhury. 2020. Methods in predictive techniques for mental health status on social media: a critical review. NPJ (2020).

[12]

Glen Coppersmith, Ryan Leary, Patrick Crutchley, and Alex Fine. 2018. Natural language processing of social media as screening for suicide risk. Biomedical informatics insights, Vol. 10 (2018), 1178222618792860.

[13]

Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. In AAAI weblogs and social media 2013 .

[14]

Munmun De Choudhury, Emre Kiciman, Mark Dredze, Glen Coppersmith, and Mrinal Kumar. 2016. Discovering shifts to suicidal ideation from mental health content in social media. In Proceedings CHI 2016. 2098--2110.

Digital Library

[15]

Raul Diaz and Amit Marathe. 2019. Soft labels for ordinal regression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition .

[16]

Christopher W Drapeau and John L McIntosh. 2020. USA suicide 2018: Official final data. (2020).

[17]

Jacob Eisenstein, Amr Ahmed, and Eric P Xing. 2011. Sparse additive generative models of text. (2011).

[18]

Casey Fiesler and Nicholas Proferes. 2018. 'Participant' perceptions of Twitter research ethics. Social Media

[19]

Society, Vol. 4, 1 (2018), 2056305118763366.

[20]

Eibe Frank and Mark Hall. 2001. A Simple Approach to Ordinal Classification. In Machine Learning: ECML 2001, Luc De Raedt and Peter Flach (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 145--156.

[21]

Joseph C Franklin, Jessica D Ribeiro, Kathryn R Fox, Kate H Bentley, Evan M Kleiman, Xieyining Huang, Adam C Jaroszewski, Bernard P Chang, and Matthew K Nock. 2017. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychological bulletin (2017).

[22]

Manas Gaur, Amanuel Alambo, Joy Prakash Sain, Ugur Kursuncu, Krishnaprasad Thirunarayan, Ramakanth Kavuluru, Amit Sheth, Randy Welton, and Jyotishman Pathak. 2019. Knowledge-aware assessment of severity of suicide risk for early intervention. In The World Wide Web Conference. 514--525.

Digital Library

[23]

Jeffrey J Glenn, Alicia L Nobles, Laura E Barnes, and Bethany A Teachman. 2019. Can Text Messages Identify Suicide Risk in Real Time? A Within-Subjects Pilot Examination of Temporally Sensitive Markers of Suicide Risk. Clinical Psychological Science (2019), 2167702620906146.

[24]

Robert N Golden, Carla Weiland, and Fred Peterson. 2009. The truth about illness and disease .Infobase Publishing.

[25]

Judith Rich Harris. 2010. No two alike: Human nature and human individuality .WW Norton & Company.

[26]

Holly Hedegaard, Sally C Curtin, and Margaret Warner. 2020. Increase in suicide mortality in the United States, 1999--2018. (2020).

[27]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput., Vol. 9, 8 (Nov. 1997), 1735--1780.

Digital Library

[28]

Dirk Hovy and Shannon L Spruit. 2016. The social impact of natural language processing. In Proceedings of ACL 2016 (Volume 2: Short Papers). 591--598.

[29]

Honor Hsin, John Torous, and Laura Roberts. 2016. An Adjuvant Role for Mobile Health in Psychiatry. JAMA Psychiatry, Vol. 73, 2 (Feb. 2016), 103.

[30]

Nicholas C Jacobson, Ashley Walton, Alexander J Millner, Garth Coombs III, and Alexandra M Rodman. 2020. Ethical dilemmas posed by mobile health and machine learning in psychiatry research. Bulletin of the WHO (2020).

[31]

Shaoxiong Ji, Shirui Pan, Xue Li, Erik Cambria, Guodong Long, and Zi Huang. 2019. Suicidal ideation detection: A review of machine learning methods and applications. arXiv preprint arXiv:1910.12611 (2019).

[32]

Kathryn P Linthicum, Katherine Musacchio Schafer, and Jessica D Ribeiro. 2019. Machine learning in suicide science: Applications and ethics. Behavioral sciences & the law, Vol. 37, 3 (2019), 214--222.

[33]

David E Losada, Fabio Crestani, and Javier Parapar. 2019. Overview of eRisk 2019 early risk prediction on the internet. In International Conference of the Cross-Language Evaluation Forum for European Languages. Springer, 340--357.

Digital Library

[34]

Naoki Masuda, Issei Kurahashi, and Hiroko Onari. 2013. Suicide Ideation of Individuals in Online Social Networks. PloS one, Vol. 8 (04 2013), e62262.

[35]

Matthew Matero, Akash Idnani, Youngseo Son, Salvatore Giorgi, Huy Vu, Mohammad Zamani, Parth Limbachiya, Sharath Chandra Guntuku, and H. Andrew Schwartz. 2019. Suicide Risk Assessment with Multi-level Dual-Context Language and BERT. In Proceedings of CLPsych 2019. ACL 2019, 39--44.

[36]

Catherine M McHugh, Amy Corderoy, Christopher James Ryan, Ian B Hickie, and Matthew Michael Large. 2019. Association between suicidal ideation and suicide: meta-analyses of odds ratios, sensitivity, specificity and positive predictive value. BJPsych open, Vol. 5, 2 (2019).

[37]

Rohan Mishra, Pradyumna Prakhar Sinha, Ramit Sawhney, Debanjan Mahata, Puneet Mathur, IIIT MIDAS, and Rajiv Ratn Shah. 2019. SNAP-BATNET: Cascading Author Profiling and Social Network Graphs for Suicide Ideation Detection on Social Media. NAACL HLT 2019 (2019), 147.

[38]

John L Oliffe, John S Ogrodniczuk, Joan L Bottorff, Joy L Johnson, and Kristy Hoyak. 2012. 'You feel like you can't live anymore': Suicide from the perspectives of Canadian men who experience depression. Social science & medicine (2012).

[39]

James Overholser. 2003. Predisposing factors in suicide attempts: life stressors. In Evaluating and treating adolescent suicide attempters. Elsevier, 41--52.

[40]

John Pestian, Henry Nasrallah, Pawel Matykiewicz, Aurora Bennett, and Antoon Leenaars. 2010. Suicide Note Classification Using Natural Language Processing: A Content Analysis. Biomedical informatics insights, Vol. 2010 (2010).

[41]

Kelly Posner, Gregory K. Brown, Barbara Stanley, David A. Brent, Kseniya V. Yershova, Maria A. Oquendo, Glenn W. Currier, Glenn A. Melvin, Laurence Greenhill, Sa Shen, and J. John Mann. 2011. The ColumbiatextendashSuicide Severity Rating Scale: Initial Validity and Internal Consistency Findings From Three Multisite Studies With Adolescents and Adults. AJP (2011).

[42]

Jo Robinson, Georgina Cox, Eleanor Bailey, Sarah Hetrick, Maria Rodrigues, Steve Fisher, and Helen Herrman. 2016. Social media and suicide prevention: a systematic review. Early intervention in psychiatry, Vol. 10, 2 (2016), 103--121.

[43]

Ramit Sawhney, Harshit Joshi, Lucie Flek, and Rajiv Ratn Shah. 2021. PHASE: Learning Emotional Phase-aware Representations for Suicide Ideation Detection on Social Media. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics .

[44]

Ramit Sawhney, Harshit Joshi, Saumya Gandhi, and Rajiv Ratn Shah. 2020. A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 7685--7697. https://doi.org/10.18653/v1/2020.emnlp-main.619

[45]

Ramit Sawhney, Prachi Manchanda, Raj Singh, and Swati Aggarwal. 2018. A computational approach to feature extraction for identification of suicidal ideation in tweets. In Proceedings of ACL 2018, Student Research Workshop. 91--98.

[46]

Han-Chin Shing, Philip Resnik, and Douglas W Oard. 2020. A Prioritization Model for Suicidality Risk Assessment. In Proceedings of ACL 2020 . 8124--8137.

[47]

Pradyumna Prakhar Sinha, Rohan Mishra, Ramit Sawhney, Debanjan Mahata, Rajiv Ratn Shah, and Huan Liu. 2019. # suicidal-A Multipronged Approach to Identify and Explore Suicidal Ideation in Twitter. In Proceedings of CIKM 2019 .

Digital Library

[48]

Hajime Sueki. 2015. The association of suicide-related Twitter use with suicidal behaviour: A cross-sectional study of young internet users in Japan. Journal of Affective Disorders, Vol. 170 (Jan. 2015), 155--160. https://doi.org/10.1016/j.jad.2014.08.047

[49]

Truyen Tran, Dinh Phung, Wei Luo, Richard Harvey, Michael Berk, and Svetha Venkatesh. 2013. An integrated framework for suicide risk prediction. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining . 1410--1418.

Digital Library

[50]

Ayah Zirikly, Philip Resnik, Özlem Uzuner, and Kristy Hollingshead. 2019. CLPsych 2019 Shared Task: Predicting the Degree of Suicide Risk in Reddit Posts. In Proceedings of CLPsych 2019. ACL 2019.

Cited By

Cabral RHan SPoon JNenadic G(2024)MM-EMOG: Multi-Label Emotion Graph Representation for Mental Health Classification on Social MediaRobotics10.3390/robotics1303005313:3(53)Online publication date: 18-Mar-2024
https://doi.org/10.3390/robotics13030053
Yan YLi JLiu XLi QYu N(2024)Identifying Reddit Users at a High Risk of Suicide and Their Linguistic Features During the COVID-19 Pandemic: Growth-Based Trajectory ModelJournal of Medical Internet Research10.2196/4890726(e48907)Online publication date: 8-Aug-2024
https://doi.org/10.2196/48907
Naseem UDunn AKhushi MKim JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Vaccine Misinformation Detection in X using Cooperative Multimodal FrameworkProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681422(4034-4042)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681422
Show More Cited By

Index Terms

Towards Ordinal Suicide Ideation Detection on Social Media
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
2. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Social media

Recommendations

Social Media is not a Health Proxy: Differences Between Social Media and Electronic Health Record Reports of Post-COVID Symptoms
CSCW

The COVID-19 pandemic transformed many aspects of health and daily life. A subset of people who were infected with the virus have ongoing chronic health issues that range in type of symptom and severity. In this study, we conducted a qualitative ...
Suicide Ideation Detection on Social Networks: Short Literature Review
Abstract
Social networking platforms will be a tool for people to share their positive and negative emotions. Suicide ideation detection is one of the important fields in social network analysis. Exploiting the data published on social networking platforms ...
Detecting and Characterizing Mental Health Related Self-Disclosure in Social Media
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

Self-disclosure is an important element facilitating improved psychological wellbeing in individuals with mental illness. As social media is increasingly adopted in health related discourse, we examine how these new platforms might be allowing honest ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

March 2021

1192 pages

ISBN:9781450382977

DOI:10.1145/3437963

General Chairs:
Liane Lewin-Eytan
Amazon, Israel
,
David Carmel
Amazon, Israel
,
Elad Yom-Tov
Microsoft, Israel
,
Program Chairs:
Eugene Agichtein
Emory University and Amazon, USA
,
Evgeniy Gabrilovich
Google Health, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM '21

Sponsor:

WSDM '21: The Fourteenth ACM International Conference on Web Search and Data Mining

March 8 - 12, 2021

Virtual Event, Israel

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
1,378
Total Downloads

Downloads (Last 12 months)213
Downloads (Last 6 weeks)19

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cabral RHan SPoon JNenadic G(2024)MM-EMOG: Multi-Label Emotion Graph Representation for Mental Health Classification on Social MediaRobotics10.3390/robotics1303005313:3(53)Online publication date: 18-Mar-2024
https://doi.org/10.3390/robotics13030053
Yan YLi JLiu XLi QYu N(2024)Identifying Reddit Users at a High Risk of Suicide and Their Linguistic Features During the COVID-19 Pandemic: Growth-Based Trajectory ModelJournal of Medical Internet Research10.2196/4890726(e48907)Online publication date: 8-Aug-2024
https://doi.org/10.2196/48907
Naseem UDunn AKhushi MKim JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Vaccine Misinformation Detection in X using Cooperative Multimodal FrameworkProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681422(4034-4042)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681422
Kaur RBhatia MKumar A(2024)Am I Hurt?: Evaluating Psychological Pain Detection in Hindi Text Using Transformer-based ModelsACM Transactions on Asian and Low-Resource Language Information Processing10.1145/365020623:8(1-17)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650206
Garg M(2024)Towards Mental Health Analysis in Social Media for Low-resourced LanguagesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/363876123:3(1-22)Online publication date: 9-Mar-2024
https://dl.acm.org/doi/10.1145/3638761
Cabral RLuo SPoon JHan SSerra ESpezzano F(2024)3M-Health: Multimodal Multi-Teacher Knowledge Distillation for Mental Health DetectionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679635(152-162)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679635
Naseem UKhushi MKim JDunn A(2024)Hybrid Text Representation for Explainable Suicide Risk Identification on Social MediaIEEE Transactions on Computational Social Systems10.1109/TCSS.2022.318498411:4(4663-4672)Online publication date: Aug-2024
https://doi.org/10.1109/TCSS.2022.3184984
Toliya NNagarathna N(2024)Leveraging Online Social Content for Early Detection of Suicidal Ideation: A Multi-Modal Deep Learning Approach2024 International Conference on Emerging Technologies in Computer Science for Interdisciplinary Applications (ICETCS)10.1109/ICETCS61022.2024.10544279(1-7)Online publication date: 22-Apr-2024
https://doi.org/10.1109/ICETCS61022.2024.10544279
Garg M(2024)WellXplain: Wellness concept extraction and classification in Reddit posts for mental health analysisKnowledge-Based Systems10.1016/j.knosys.2023.111228284(111228)Online publication date: Jan-2024
https://doi.org/10.1016/j.knosys.2023.111228
Kumar AChopra SAggarwal S(2024)Assessing Suicidal Tendencies on Twitter Using BERTicleInformation Systems for Intelligent Systems10.1007/978-981-99-8612-5_3(23-31)Online publication date: 27-Feb-2024
https://doi.org/10.1007/978-981-99-8612-5_3
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents