research-article

Unintended Bias in Misogyny Detection

Authors:

Claudia Volpetti,

Elisabetta FersiniAuthors Info & Claims

WI '19: IEEE/WIC/ACM International Conference on Web Intelligence

Pages 149 - 155

https://doi.org/10.1145/3350546.3352512

Published: 14 October 2019 Publication History

Abstract

During the last years, the phenomenon of hate against women increased exponentially especially in online environments such as microblogs. Although this alarming phenomenon has triggered many studies both from computational linguistic and machine learning points of view, less effort has been spent to analyze if those misogyny detection models are affected by an unintended bias. This can lead the models to associate unreasonably high misogynous scores to a non-misogynous text only because it contains certain terms, called identity terms. This work is the first attempt to address the problem of measuring and mitigating unintended bias in machine learning models trained for the misogyny detection task. We propose a novel synthetic test set that can be used as evaluation framework for measuring the unintended bias and different mitigation strategies specific for this task. Moreover, we provide a misogyny detection model that demonstrate to obtain the best classification performance in the state-of-the-art. Experimental results on recently introduced bias metrics confirm the ability of the bias mitigation treatment to reduce the unintended bias of the proposed misogyny detection model.

References

[1]

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/Software available from tensorflow.org.

[2]

Maria Anzovino, Elisabetta Fersini, and Paolo Rosso. 2018. Automatic Identification and Classification of Misogynistic Language on Twitter. In Proceedings of the 23rd International Conference on Applications of Natural Language to Information Systems (NLDB 2018)(Lecture Notes in Computer Science), Max Silberztein, Faten Atigui, Elena Kornyshova, Elisabeth Métais, and Farid Meziane (Eds.), Vol. 10859. Springer, 57–64.

Digital Library

[3]

Angelo Basile and Chiara Rubagotti. 2018. CrotoneMilano for AMI at Evalita2018. A Performant, Cross-lingual Misogyny Detection System. In Proceedings of 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2018). CEUR.org.

[4]

Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter. In Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval 2019). Association for Computational Linguistics, 54–63.

[5]

Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H. Chi. 2017. Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations. arXiv preprint arXiv:1707.00075 (jun 2017). arxiv:1707.00075

[6]

Daniel Borkan, Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Limitations of Pinned AUC for Measuring Unintended Bias. arXiv preprint arXiv:1903.02088(2019).

[7]

Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification. In Companion of The 2019 World Wide Web Conference (WWW 2019). ACM.

[8]

Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018. Universal Sentence Encoder for English. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP 2018). Association for Computational Linguistics, 169–174.

[9]

François Chollet 2015. Keras. https://keras.io.

[10]

Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. ACM, 67–73.

Digital Library

[11]

Yanai Elazar and Yoav Goldberg. 2018. Adversarial Removal of Demographic Attributes from Text Data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018). Association for Computational Linguistics, 11–21.

[12]

Elisabetta Fersini, Debora Nozza, and Paolo Rosso. 2018. Overview of the EVALITA 2018 Task on Automatic Misogyny Identification (AMI). In Proceedings of Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2018). CEUR-WS.org.

[13]

Elisabetta Fersini, Paolo Rosso, and Maria Anzovino. 2018. Overview of the Task on Automatic Misogyny Identification at IberEval 2018. In Proceedings of the Third Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval 2018), co-located with 34th Conference of the Spanish Society for Natural Language Processing (SEPLN 2018). CEUR-WS.org.

[14]

Simona Frenda, Bilal Ghanem, and Manuel Montes-y Gómez. 2018. Exploration of Misogyny in Spanish and English tweets. In Proceedings of the Third Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval 2018), co-located with 34th Conference of the Spanish Society for Natural Language Processing (SEPLN 2018), Vol. 2150. CEUR-WS.org, 260–267.

[15]

Iakes Goenaga, Aitziber Atutxa, Koldo Gojenola, Arantza Casillas, Arantza Díaz de Ilarraza, Nerea Ezeiza, Maite Oronoz, Alicia Pérez, and Olatz Perez-de-Viñaspre. 2018. Automatic Misogyny Identification Using Neural Networks. In Proceedings of the 3rd Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval 2018), co-located with 34th Conference of the Spanish Society for Natural Language Processing (SEPLN 2018). CEUR-WS.org.

[16]

Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In Advances in Neural Information Processing Systems. 3315–3323.

[17]

Sarah Hewitt, Thanassis Tiropanis, and Christian Bokhove. 2016. The Problem of identifying Misogynist Language on Twitter (and other online social spaces). In Proceedings of the 8th ACM Conference on Web Science. ACM Press, 333–335.

Digital Library

[18]

Svetlana Kiritchenko and Saif Mohammad. 2018. Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM@NAACL-HLT 2018). Association for Computational Linguistics, 43–53.

[19]

Endang Wahyu Pamungkas, Alessandra Teresa Cignarella, Valerio Basile, and Viviana Patti. 2018. 14-ExLab@ UniTo for AMI at IberEval2018: Exploiting lexical knowledge for detecting misogyny in English and Spanish tweets. In Proceedings of the 3rd Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval 2018), co-located with 34th Conference of the Spanish Society for Natural Language Processing (SEPLN 2018), Vol. 2150. CEUR-WS.org, 234–241.

[20]

Endang Wahyu Pamungkas, Alessandra Teresa Cignarella, Valerio Basile, and Viviana Patti. 2018. Automatic Identification of Misogyny in English and Italian Tweets at EVALITA 2018 with a Multilingual Hate Lexicon. In Proceedings of 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2018). CEUR.org.

[21]

Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing Gender Bias in Abusive Language Detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018). Association for Computational Linguistics, 2799–2804.

[22]

Punyajoy Saha, Binny Mathew, Pawan Goyal, and Animesh Mukherjee. 2018. Hateminers: Detecting Hate speech against Women. arXiv preprint arXiv:1812.06700(2018).

[23]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems (NIPS 2017). 6000–6010.

[24]

Zeerak Waseem and Dirk Hovy. 2016. Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. In Proceedings of the Student Research Workshop, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (SRW@HLT-NAACL 2016). Association for Computational Linguistics, 88–93.

Cited By

Sharma DSingh ASingh V(2024)THAR- Targeted Hate Speech Against Religion: A high-quality Hindi-English code-mixed Dataset with the Application of Deep Learning Models for Automatic DetectionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/3653017Online publication date: 18-Mar-2024
https://doi.org/10.1145/3653017
Rabonato RBerton L(2024)A systematic review of fairness in machine learningAI and Ethics10.1007/s43681-024-00577-5Online publication date: 19-Sep-2024
https://doi.org/10.1007/s43681-024-00577-5
Kumari GSinha AEkbal AChatterjee AN V(2024)Enhancing the fairness of offensive memes detection models by mitigating unintended political biasJournal of Intelligent Information Systems10.1007/s10844-023-00834-962:3(735-763)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10844-023-00834-9
Show More Cited By

Index Terms

Unintended Bias in Misogyny Detection

Index terms have been assigned to the content through auto-classification.

Recommendations

Categorizing Sexism and Misogyny through Neural Approaches
Sexism, an injustice that subjects women and girls to enormous suffering, manifests in blatant as well as subtle ways. In the wake of growing documentation of experiences of sexism on the web, the automatic categorization of accounts of sexism has the ...
Protected attribute guided representation learning for bias mitigation in limited data
Abstract
In many real-world problem settings, the training data may contain certain unwanted correlations or biases. Such biases generally occur due to imbalances/skew in the data with respect to some protected attributes. When a standard deep ...
Handling Bias in Toxic Speech Detection: A Survey
Detecting online toxicity has always been a challenge due to its inherent subjectivity. Factors such as the context, geography, socio-political climate, and background of the producers and consumers of the posts play a crucial role in determining if the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WI '19: IEEE/WIC/ACM International Conference on Web Intelligence

October 2019

507 pages

ISBN:9781450369343

DOI:10.1145/3350546

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WI '19

WI '19: IEEE/WIC/ACM International Conference on Web Intelligence

October 14 - 17, 2019

Thessaloniki, Greece

Acceptance Rates

Overall Acceptance Rate 118 of 178 submissions, 66%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
710
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)8

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sharma DSingh ASingh V(2024)THAR- Targeted Hate Speech Against Religion: A high-quality Hindi-English code-mixed Dataset with the Application of Deep Learning Models for Automatic DetectionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/3653017Online publication date: 18-Mar-2024
https://doi.org/10.1145/3653017
Rabonato RBerton L(2024)A systematic review of fairness in machine learningAI and Ethics10.1007/s43681-024-00577-5Online publication date: 19-Sep-2024
https://doi.org/10.1007/s43681-024-00577-5
Kumari GSinha AEkbal AChatterjee AN V(2024)Enhancing the fairness of offensive memes detection models by mitigating unintended political biasJournal of Intelligent Information Systems10.1007/s10844-023-00834-962:3(735-763)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10844-023-00834-9
Rawat AKumar SSamant S(2024)Hate speech detection in social media: Techniques, recent trends, and future challengesWIREs Computational Statistics10.1002/wics.164816:2Online publication date: 11-Mar-2024
https://doi.org/10.1002/wics.1648
Yadav SKaushik ASharma S(2023)Comprehensive Analysis of the Artificial Intelligence Approaches for Detecting Misogynistic Mixed-Code Online Content in South Asian CountriesCyberfeminism and Gender Violence in Social Media10.4018/978-1-6684-8893-5.ch025(350-368)Online publication date: 29-Sep-2023
https://doi.org/10.4018/978-1-6684-8893-5.ch025
Tontodimamma AAnzani SStranisci MBasile VIgnazzi EFontanella L(2023)An experimental annotation task to investigate annotators’ subjectivity in a Misogyny datasetASA 2022 Data-Driven Decision Making10.36253/979-12-215-0106-3.49(281-286)Online publication date: 2023
https://doi.org/10.36253/979-12-215-0106-3.49
Garg TMasud SSuresh TChakraborty T(2023)Handling Bias in Toxic Speech Detection: A SurveyACM Computing Surveys10.1145/358049455:13s(1-32)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3580494
Bendale BShirke S(2023)Computer Science Responses to Abuse Toward Women and Children2023 1st International Conference on Cognitive Computing and Engineering Education (ICCCEE)10.1109/ICCCEE55951.2023.10424634(1-8)Online publication date: 27-Apr-2023
https://doi.org/10.1109/ICCCEE55951.2023.10424634
Almaliki M(2023)Cyberhate Dissemination: A Systematic Literature MapIEEE Access10.1109/ACCESS.2023.332625411(117385-117392)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3326254
Donald AGalanopoulos ACurry EMuñoz EUllah IWaskow MDabrowski MKalra M(2023)Bias Detection for Customer Interaction Data: A Survey on Datasets, Methods, and ToolsIEEE Access10.1109/ACCESS.2023.327675711(53703-53715)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3276757
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents