skip to main content
10.1145/3530190.3534792acmconferencesArticle/Chapter ViewAbstractPublication PagescompassConference Proceedingsconference-collections
research-article

The Six Conundrums of Building and Deploying Language Technologies for Social Good

Published: 29 June 2022 Publication History

Abstract

Deployment of speech and language technology for social good (LT4SG), especially those targeted at the welfare of marginalized communities and speakers of low-resource and under-served languages, has been a prominent theme of research within NLP, Speech and the AI communities. Many researchers, especially those working in core NLP/Speech domains, rely on a combination of individual expertise, experiences or ad hoc surveys for prioritizing between language technologies that provide social good to the end-users. This has been criticized by several scholars who argue that it is critical to include the target community during the LT’s design and development process. However, prioritization of communities, languages, technologies and design approaches presents a very large set of complex challenges to the technologists, for which there are no simple or off-the-shelf solutions. In this position paper, we distill our experiential insights into six fundamental conundrums that technologists face and must resolve while deciding which LT technology to build for which community, and by using what approach. We discuss that at the root of these conundrums lie certain fundamental ethical problems of a digital-divide that can be overcome only by resolving deeper ethical dilemmas of distributive justice. We urge the community to reflect on these conundrums and leverage shared experiential insights to reconcile the intent of broadly, any Technology for Social Good, with the ground realities of its deployment.

Supplementary Material

MP4 File (COMPASS_Paper_Session1_2022-06-29_ChoudhuryM.mp4)
Conference Presentation Recording 2022-06-29

References

[1]
Oscar Alvarado, Hendrik Heuer, Vero Vanden Abeele, Andreas Breiter, and Katrien Verbert. 2020. Middle-Aged Video Consumers’ Beliefs About Algorithmic Recommendations on YouTube. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2(2020), 1–24.
[2]
Valeriya Azarova, Jed Cohen, Christina Friedl, and Johannes Reichl. 2019. Designing local renewable energy communities to increase social acceptance: Evidence from a choice experiment in Austria, Germany, Italy, and Switzerland. Energy Policy 132(2019), 1176–1183. https://doi.org/10.1016/j.enpol.2019.06.067
[3]
Fan Bai, Alan Ritter, and Wei Xu. 2021. Pre-train or annotate? domain adaptation with a constrained budget. arXiv preprint arXiv:2109.04711(2021).
[4]
Michálle E. Mor Barak. 2020. The Practice and Science of Social Good: Emerging Paths to Positive Social Impact. Research on Social Work Practice 30 (2020), 139 – 150.
[5]
Christoph Becker, Ann Light, Chris Frauenberger, Dawn Walker, Victoria Palacin, Syed Ishtiaque Ahmed, Rachel Charlotte Smith, Pedro Reynolds Cuéllar, and David Nemer. 2020. Computing professionals for social responsibility: The past, present and future values of participatory design. In Proceedings of the 16th Participatory Design Conference 2020-Participation (s) Otherwise-Volume 2. 181–184.
[6]
Steven Bird. 2020. Decolonising Speech and Language Technology. In Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, Barcelona, Spain (Online), 3504–3519. https://doi.org/10.18653/v1/2020.coling-main.313
[7]
Damián Blasi, Antonios Anastasopoulos, and Graham Neubig. 2021. Systematic Inequalities in Language Technology Performance across the World’s Languages. arXiv preprint arXiv:2110.06733(2021).
[8]
Rowena L. Briones, Beth Kuch, Brooke Fisher Liu, and Yan Jin. 2011. Keeping up with the digital age: How the American Red Cross uses social media to build relationships. Public Relations Review 37, 1 (2011), 37–43. https://doi.org/10.1016/j.pubrev.2010.12.006
[9]
JM Buchanan. 1972. The Samaritan’s Dilemma, reprinted in: Buchanan, JM (1977): Freedom in Constitutional Contract.
[10]
G. Campitelli and F. Gobet. 2010. Herbert Simon’s Decision-Making Approach: Investigation of Cognitive Processes in Experts. Review of General Psychology 14 (2010), 354 – 364.
[11]
Tommaso Caselli, Roberto Cibin, Costanza Conforti, Enrique Ecinas, and Maurizio Teli. 2021. Guiding Principles for Participatory Design-inspired Natural Language Processing. In Proceedings of the 1st Workshop on NLP for Positive Impact. Association for Computational Linguistics, 27–35.
[12]
Long Cheng, Xuewu Chen, Jonas De Vos, Xinjun Lai, and Frank Witlox. 2019. Applying a random forest method approach to model travel mode choice behavior. Travel Behaviour and Society 14 (2019), 1–10. https://doi.org/10.1016/j.tbs.2018.09.002
[13]
Dimple Choudhury. 2020. Multilingual education and the mother tongue: A survey of two endangered languages of Arunachal Pradesh. Language and language teaching 8, 18 (2020), 18–22.
[14]
M. Choudhury and Amit Deshpande. 2021. How Linguistically Fair Are Multilingual Pre-Trained Language Models?. In AAAI.
[15]
Yi-Shih Chung and Yu-Chiun Chiou. 2017. Willingness-to-pay for a bus fare reform: A contingent valuation approach with multiple bound dichotomous choices. Transportation Research Part A: Policy and Practice 95 (2017), 289–304. https://doi.org/10.1016/j.tra.2016.11.018
[16]
S. V. Ciriacy-Wantrup. 1947. Capital Returns from Soil-Conservation Practices. Journal of Farm Economics 29, 4 (1947), 1181–1196. http://www.jstor.org/stable/1232747
[17]
James S. Coleman. 1988. Social Capital in the Creation of Human Capital. Amer. J. Sociology 94(1988), S95–S120. http://www.jstor.org/stable/2780243
[18]
Josh Cowls, Andreas Tsamados, Mariarosaria Taddeo, and Luciano Floridi. 2021. A definition, benchmark and database of AI for social good initiatives. Nature Machine Intelligence 3, 2 (01 Feb 2021), 111–115. https://doi.org/10.1038/s42256-021-00296-0
[19]
Paul Dourish, Christopher Lawrence, Tuck Wah Leong, and Greg Wadley. 2020. On being iterated: The affective demands of design participation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–11.
[20]
Kawin Ethayarajh and Dan Jurafsky. 2020. Utility is in the Eye of the User: A Critique of NLP Leaderboards. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 4846–4853. https://doi.org/10.18653/v1/2020.emnlp-main.393
[21]
Luciano Floridi, Josh Cowls, Thomas C. King, and Mariarosaria Taddeo. 2020. How to Design AI for Social Good: Seven Essential Factors. Science and Engineering Ethics 26, 3 (01 Jun 2020), 1771–1796. https://doi.org/10.1007/s11948-020-00213-5
[22]
Paula Fortuna, Laura Pérez-Mayos, Ahmed Ghassan Tawfiq AbuRa’ed, Juan Soler-Company, and L. Wanner. 2021. Cartography of Natural Language Processing for Social Good (NLP4SG): Searching for Definitions, Statistics and White Spots. In NLP4POSIMPACT.
[23]
Colin Green and Karen Gerard. 2009. Exploring the social value of health-care interventions: A stated preference discrete choice experiment. Health economics 18 (08 2009), 951–76. https://doi.org/10.1002/hec.1414
[24]
Hendrik Heuer and Daniel Buschek. 2021. Methods for the Design and Evaluation of HCI+ NLP Systems. arXiv preprint arXiv:2102.13461(2021).
[25]
Nanna Inie and Leon Derczynski. 2021. An IDR Framework of Opportunities and Barriers between HCI and NLP. In Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing. Association for Computational Linguistics, Online, 101–108. https://aclanthology.org/2021.hcinlp-1.16
[26]
Zhijing Jin, Geeticka Chauhan, Brian Tse, Mrinmaya Sachan, and Rada Mihalcea. 2021. How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics, Online, 3099–3113. https://doi.org/10.18653/v1/2021.findings-acl.273
[27]
Pratik Joshi, Christain Barnes, Sebastin Santy, Simran Khanuja, Sanket Shah, Anirudh Srinivasan, Satwik Bhattamishra, Sunayana Sitaram, Monojit Choudhury, and Kalika Bali. 2019. Unsung challenges of building and deploying language technologies for low resource language communities. arXiv preprint arXiv:1912.03457(2019).
[28]
Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The State and Fate of Linguistic Diversity and Inclusion in the NLP World. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 6282–6293. https://doi.org/10.18653/v1/2020.acl-main.560
[29]
Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A Smith, and Daniel S Weld. 2021. Genie: A leaderboard for human-in-the-loop evaluation of text generation. arXiv preprint arXiv:2101.06561(2021).
[30]
Arun Khatri-Chhetri, P.K. Aggarwal, P.K. Joshi, and S. Vyas. 2017. Farmers’ prioritization of climate-smart agriculture (CSA) technologies. Agricultural Systems 151(2017), 184–191. https://doi.org/10.1016/j.agsy.2016.10.005
[31]
Amanda Lazar, Jessica L Feuston, Caroline Edasis, and Anne Marie Piper. 2018. Making as expression: Informing design with people with complex communication needs through art therapy. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–16.
[32]
Kristen Lovejoy and Gregory D. Saxton. 2012. Information, Community, and Action: How Nonprofit Organizations Use Social Media*. Journal of Computer-Mediated Communication 17, 3 (2012), 337–353. https://doi.org/10.1111/j.1083-6101.2012.01576.x arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1083-6101.2012.01576.x
[33]
Benjamin Marie, Atsushi Fujita, and Raphael Rubino. 2021. Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 7297–7306. https://doi.org/10.18653/v1/2021.acl-long.566
[34]
Devansh Mehta, Sebastin Santy, Ramaravind Kommiya Mothilal, Brij Mohan Lal Srivastava, Alok Sharma, Anurag Shukla, Vishnu Prasad, Amit Sharma, Kalika Bali, 2020. Learnings from Technological Interventions in a Low Resource Language: A Case-Study on Gondi. arXiv preprint arXiv:2004.10270(2020).
[35]
Anna Merino. 2003. Eliciting consumers preferences using stated preference discrete choice models: Contingent ranking versus choice experiment. Economics Working Papers 705. Department of Economics and Business, Universitat Pompeu Fabra. https://ideas.repec.org/p/upf/upfgen/705.html
[36]
Robert K Merton. 1968. The Matthew effect in science: The reward and communication systems of science are considered. Science 159, 3810 (1968), 56–63.
[37]
Oussama Metatla, Alison Oldfield, Taimur Ahmed, Antonis Vafeas, and Sunny Miglani. 2019. Voice user interfaces in schools: Co-designing for inclusion with visually-impaired and sighted pupils. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–15.
[38]
Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Tajudeen Kolawole, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Hassan Muhammad, Salomon Kabongo, Salomey Osei, 2020. Participatory research for low-resourced machine translation: A case study in african languages. arXiv preprint arXiv:2010.02353(2020).
[39]
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 311–318. https://doi.org/10.3115/1073083.1073135
[40]
Soya Park, April Yi Wang, Ban Kawas, Q Vera Liao, David Piorkowski, and Marina Danilevsky. 2021. Facilitating knowledge sharing from domain experts to data scientists for building nlp models. In 26th International Conference on Intelligent User Interfaces. 585–596.
[41]
Jennifer Pierre, Roderic Crooks, Morgan Currie, Britt Paris, and Irene Pasquetto. 2021. Getting Ourselves Together: Data-centered participatory design research & epistemic burden. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–11.
[42]
John Rawls. 1971. A Theory of Justice. The Belknap Press of Harvard University Press.
[43]
Ananya B Sai, Akash Kumar Mohankumar, and Mitesh M Khapra. 2020. A survey of evaluation metrics used for NLG systems. arXiv preprint arXiv:2008.12009(2020).
[44]
Sebastin Santy, Kalika Bali, Monojit Choudhury, Sandipan Dandapat, Tanuja Ganu, Anurag Shukla, Jahanvi Shah, and Vivek Seshadri. 2021. Language Translation as a Socio-Technical System:Case-Studies of Mixed-Initiative Interactions. In ACM SIGCAS Conference on Computing and Sustainable Societies (Virtual Event, Australia) (COMPASS ’21). Association for Computing Machinery, New York, NY, USA, 156–172. https://doi.org/10.1145/3460112.3471954
[45]
Amartya Sen. 1985. Social Choice and Justice: A Review Article. Journal of Economic Literature 23, 4 (1985), 1764–1776. http://www.jstor.org/stable/2725708
[46]
H.A. Simon. 1957. Models of Man: Social and Rational; Mathematical Essays on Rational Human Behavior in Society Setting. Wiley. https://books.google.com/books?id=_w1gAAAAIAAJ
[47]
Jesper Simonsen and Toni Robertson (Eds.). 2012. Routledge International Handbook of Participatory Design. Routledge.
[48]
Adam Smith. 2021. An Inquiry into the Nature and Causes of the Wealth of Nations:. University of Chicago Press. https://doi.org/
[49]
Michael F Thomas. 2012. Building community participation into documentation design: lessons learned in Sakun (Sukur). Language Documentation and Description 11 (2012), 59–71.
[50]
Greg Walsh, Elizabeth Foss, Jason Yip, and Allison Druin. 2013. FACIT PD: a framework for analysis and creation of intergenerational techniques for participatory design. In proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2893–2902.
[51]
Ning Wang, Linhao Tang, and Huizhong Pan. 2017. Effectiveness of policy incentives on electric vehicle acceptance in China: A discrete choice analysis. Transportation Research Part A: Policy and Practice 105 (2017), 210–218. https://doi.org/10.1016/j.tra.2017.08.009
[52]
Matthew Wiswall and Basit Zafar. 2018. Preference for the Workplace, Investment in Human Capital, and Gender. The Quarterly Journal of Economics 133, 1 (2018), 457–507. https://EconPapers.repec.org/RePEc:oup:qjecon:v:133:y:2018:i:1:p:457-507.
[53]
A. Zanolini, Kombatende Sikombe, I. Sikazwe, I. Eshun-Wilson, Paul Somwe, Carolyn Bolton Moore, S. Topp, N. Czaicki, L. Beres, C. Mwamba, N. Padian, C. Holmes, and E. Geng. 2018. Understanding preferences for HIV care and treatment in Zambia: Evidence from a discrete choice experiment among patients who have been lost to follow-up. PLoS Medicine 15(2018).

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
COMPASS '22: Proceedings of the 5th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies
June 2022
710 pages
ISBN:9781450393478
DOI:10.1145/3530190
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 June 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. community-centric development
  2. language technology
  3. socio-technical deployment

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

COMPASS '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 25 of 50 submissions, 50%

Upcoming Conference

COMPASS '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)48
  • Downloads (Last 6 weeks)8
Reflects downloads up to 16 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media