research-article

Open access

When is Machine Learning Data Good?: Valuing in Public Health Datafication

Authors: Divy Thakkar, Azra Ismail, Pratyush Kumar, Alex Hanna, Nithya Sambasivan, Neha KumarAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Article No.: 322, Pages 1 - 16

https://doi.org/10.1145/3491102.3501868

Published: 29 April 2022 Publication History

All formats PDF

Abstract

Data-driven approaches that form the foundation of advancements in machine learning (ML) are powered in large part by human infrastructures that enable the collection of large datasets. We study the movement of data through multiple stages of data processing in the context of public health in India, examining the data work performed by frontline health workers, data stewards, and ML developers. We conducted interviews with these stakeholders to understand their varied perspectives on valuing data across stages, working with data to attain this value, and challenges arising throughout. We discuss the tensions in valuing and how they might be addressed, as we emphasize the need for improved transparency and accountability when data are transformed from one stage of processing to the next.

Supplemental Material

MP4 File

Talk Video

Transcript for: Talk Video

References

[1]

Michael D Abràmoff, Philip T Lavin, Michele Birch, Nilay Shah, and James C Folk. 2018. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ digital medicine 1, 1 (2018), 1–8.

[2]

Fábio S Aguiar, Rodrigo C Torres, João VF Pinto, Afrânio L Kritski, José M Seixas, and Fernanda CQ Mello. 2016. Development of two artificial neural network models to support the diagnosis of pulmonary tuberculosis in hospitalized patients in Rio de Janeiro, Brazil. Medical & biological engineering & computing 54, 11(2016), 1751–1759.

[3]

Syed Ishtiaque Ahmed, Md Romael Haque, Shion Guha, Md Rashidujjaman Rifat, and Nicola Dell. 2017. Privacy, security, and surveillance in the Global South: A study of biometric mobile SIM registration in Bangladesh. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 906–918.

Digital Library

[4]

Karen S Baker and Helena Karasti. 2018. Data care and its politics: Designing for local collective data management as a neglected thing. In Proceedings of the 15th Participatory Design Conference: Full Papers-Volume 1. 1–12.

Digital Library

[5]

Agathe Balayn, Bogdan Kulynych, and Seda Guerses. 2021. Exploring Data Pipelines through the Process Lens: a Reference Model forComputer Vision. arXiv preprint arXiv:2107.01824(2021).

[6]

Carlo Batini, Cinzia Cappiello, Chiara Francalanci, and Andrea Maurino. 2009. Methodologies for data quality assessment and improvement. ACM computing surveys (CSUR) 41, 3 (2009), 1–52.

[7]

Amna Batool, Kentaro Toyama, Tiffany Veinot, Beenish Fatima, and Mustafa Naseem. 2021. Detecting Data Falsification by Front-line Development Workers: A Case Study of Vaccination in Pakistan. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[8]

Emma Beede, Elizabeth Baylor, Fred Hersch, Anna Iurchenko, Lauren Wilcox, Paisan Ruamviboonsuk, and Laura M Vardoulakis. 2020. A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[9]

Kaustav Bera, Kurt A Schalper, David L Rimm, Vamsidhar Velcheti, and Anant Madabhushi. 2019. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nature reviews Clinical oncology 16, 11 (2019), 703–715.

[10]

Patrick Biernacki and Dan Waldorf. 1981. Snowball sampling: Problems and techniques of chain referral sampling. Sociological methods & research 10, 2 (1981), 141–163.

[11]

Alan Borning and Michael Muller. 2012. Next steps for value sensitive design. In Proceedings of the SIGCHI conference on human factors in computing systems. 1125–1134.

Digital Library

[12]

Mark Bovens. 2007. Analysing and assessing accountability: A conceptual framework. European law journal 13, 4 (2007), 447–468.

[13]

Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77–91.

[14]

Massimo Buscema, Masoud Asadi-Zeydabadi, Weldon Lodwick, Alphonse Nde Nembot, Alvin Bronstein, and Francis Newman. 2020. Analysis of the Ebola Outbreak in 2014 and 2018 in West Africa and Congo by Using Artificial Adaptive Systems. Applied Artificial Intelligence 34, 8 (2020), 597–617.

[15]

Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. ” Hello AI”: Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proceedings of the ACM on Human-computer Interaction 3, CSCW(2019), 1–24.

Digital Library

[16]

HC Stephen Chan, Hanbin Shan, Thamani Dahoun, Horst Vogel, and Shuguang Yuan. 2019. Advancing drug discovery via artificial intelligence. Trends in pharmacological sciences 40, 8 (2019), 592–604.

[17]

Irene Y Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. 2020. Ethical Machine Learning in Health. arXiv preprint arXiv:2009.10576(2020).

[18]

Marika Cifor, Patricia Garcia, TL Cowan, Jasmine Rault, Tonia Sutherland, Anita Say Chan, Jennifer Rode, Anna Lauren Hoffmann, Niloufar Salehi, and Lisa Nakamura. 2019. Feminist data manifest-no.

[19]

Nicola Dell, Trevor Perrier, Neha Kumar, Mitchell Lee, Rachel Powers, and Gaetano Borriello. 2015. Digital Workflows in Global Development Organizations. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 1659–1669.

Digital Library

[20]

Emily Denton, Alex Hanna, Razvan Amironesei, Andrew Smart, Hilary Nicole, and Morgan Klaus Scheuerman. 2020. Bringing the People Back In: Contesting Benchmark Machine Learning Datasets. arXiv preprint arXiv:2007.07399(2020).

[21]

Brian DeRenzi, Nicola Dell, Jeremy Wacksman, Scott Lee, and Neal Lesh. 2017. Supporting Community Health Workers in India through Voice-and Web-Based Feedback. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 2770–2781.

Digital Library

[22]

Bhavna Dhingra and Ashok Kumar Dutta. 2011. National rural health mission. The Indian Journal of Pediatrics 78, 12 (2011), 1520–1526.

[23]

Catherine D’Ignazio and Lauren F Klein. 2020. Data feminism. MIT Press.

[24]

Paul Dourish and Edgar Gómez Cruz. 2018. Datafication and data fiction: Narrating data and narrating with data. Big Data & Society 5, 2 (2018), 2053951718784083.

[25]

Chris Elsden, Kate Symons, Raluca Bunduchi, Chris Speed, and John Vines. 2019. Sorting out valuation in the charity shop: Designing for data-driven innovation through value translation. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–25.

Digital Library

[26]

Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.

Digital Library

[27]

Melanie Feinberg. 2017. A design perspective on data. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2952–2963.

Digital Library

[28]

Brittany Fiore-Gartland and Gina Neff. 2015. Communication, mediation, and the expectations of data: Data valences across health and wellness communities. International Journal of Communication 9 (2015), 19.

[29]

Batya Friedman, Peter Kahn, and Alan Borning. 2002. Value sensitive design: Theory and methods. University of Washington technical report2-12 (2002).

[30]

Batya Friedman, Peter H Kahn, Alan Borning, and Alina Huldtgren. 2013. Value sensitive design and information systems. In Early engagement and new technologies: Opening up the laboratory. Springer, 55–95.

[31]

R Stuart Geiger, Kevin Yu, Yanlai Yang, Mindy Dai, Jie Qiu, Rebekah Tang, and Jenny Huang. 2020. Garbage in, garbage out? do machine learning application papers in social computing report where human-labeled training data comes from?. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 325–336.

Digital Library

[32]

Hongyan Gu, Jingbin Huang, Lauren Hung, and Xiang’Anthony’ Chen. 2021. Lessons learned from designing an AI-enabled diagnosis tool for pathologists. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–25.

Digital Library

[33]

Venkat Gudivada, Amy Apon, and Junhua Ding. 2017. Data quality considerations for big data and machine learning: Going beyond data cleaning and transformations. International Journal on Advances in Software 10, 1 (2017), 1–20.

[34]

Pi Guo, Tao Liu, Qin Zhang, Li Wang, Jianpeng Xiao, Qingying Zhang, Ganfeng Luo, Zhihao Li, Jianfeng He, Yonghui Zhang, and Wenjun Ma. 2017. Developing a dengue forecast model using machine learning: A case study in China. PLoS neglected tropical diseases 11, 10 (2017), e0005973.

[35]

Philip J Guo, Sean Kandel, Joseph M Hellerstein, and Jeffrey Heer. 2011. Proactive wrangling: Mixed-initiative end-user programming of data transformation scripts. In Proceedings of the 24th annual ACM symposium on User interface software and technology. 65–74.

Digital Library

[36]

Frank Heuts and Annemarie Mol. 2013. What is a good tomato? A case of valuing in practice. Valuation Studies 1, 2 (2013), 125–146.

[37]

Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. 2019. Improving fairness in machine learning systems: What do industry practitioners need?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–16.

Digital Library

[38]

Shigao Huang, Jie Yang, Simon Fong, and Qi Zhao. 2020. Artificial intelligence in cancer diagnosis and prognosis: Opportunities and challenges. Cancer Letters 471(2020), 61–71.

[39]

Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. 2021. Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 560–575.

Digital Library

[40]

Eleanor Hutchinson, Susan Nayiga, Christine Nabirye, Lilian Taaka, and Sarah G Staedke. 2018. Data value and care value in the practice of health systems: A case study in Uganda. Social science & medicine 211 (2018), 123–130.

[41]

Marco Iansiti. 2021. The Value of Data and Its Impact on Competition. Available at SSRN (2021).

[42]

Azra Ismail and Neha Kumar. 2018. Engaging Solidarity in Data Collection Practices for Community Health. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 76.

Digital Library

[43]

Azra Ismail and Neha Kumar. 2019. Empowerment on the Margins: The Online Experiences of Community Health Workers. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 99.

Digital Library

[44]

Eun Seo Jo and Timnit Gebru. 2020. Lessons from archives: strategies for collecting sociocultural data in machine learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 306–316.

Digital Library

[45]

Severin Kacianka and Alexander Pretschner. 2021. Designing Accountable Systems. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 424–437.

Digital Library

[46]

Ramesha Karunasena, Mohammad Sarparajul Ambiya, Arunesh Sinha, Ruchit Nagar, Saachi Dalal, Divy Thakkar, and Milind Tambe. 2020. Measuring Data Collection Quality for Community Healthcare. arXiv preprint arXiv:2011.02962(2020).

[47]

Naveena Karusala, Jennifer Wilson, Phebe Vayanos, and Eric Rice. 2019. Street-Level Realities of Data Practices in Homeless Services Provision. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–23.

Digital Library

[48]

Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Bintz, Daniella Raz, and PM Krafft. 2020. Toward situated interventions for algorithmic equity: lessons from the field. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 45–55.

Digital Library

[49]

Os Keyes. 2018. The misgendering machines: Trans/HCI implications of automatic gender recognition. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 88.

Digital Library

[50]

Rob Kitchin and Tracey Lauriault. 2014. Towards critical data studies: Charting and unpacking data assemblages and their work. (2014).

[51]

Meghana Kshirsagar, Caleb Robinson, Siyu Yang, Shahrzad Gholami, Ivan Klyuzhin, Sumit Mukherjee, Md Nasir, Anthony Ortiz, Felipe Oviedo, Darren Tanner, 2021. Becoming Good at AI for Good. arXiv preprint arXiv:2104.11757(2021).

[52]

David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani. 2014. The parable of Google Flu: traps in big data analysis. Science 343, 6176 (2014), 1203–1205.

[53]

Yanni Alexander Loukissas. 2019. All data are local: Thinking critically in a data-driven society. MIT Press.

[54]

Yaoli Mao, Dakuo Wang, Michael Muller, Kush R Varshney, Ioana Baldini, Casey Dugan, and Aleksandra Mojsilović. 2019. How data scientistswork together with domain experts in scientific collaborations: To find the right answer or to ask the right question?Proceedings of the ACM on Human-Computer Interaction 3, GROUP(2019), 1–23.

[55]

Aditya Mate, Jackson A Killian, Haifeng Xu, Andrew Perrault, and Milind Tambe. 2020. Collapsing Bandits and Their Application to Public Health Interventions. arXiv preprint arXiv:2007.04432(2020).

[56]

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635(2019).

[57]

Amanda Meng, Carl DiSalvo, and Ellen Zegura. 2019. Collaborative data work towards a caring democracy. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–23.

Digital Library

[58]

Syeda Shaizadi Meraj, Razali Yaakob, Azreen Azman, Siti Nuralain Mohd Rum, Azree Shahrel, and Ahmad Nazri. 2019. Artificial Intelligence in Diagnosing Tuberculosis: A Review. Int. Journal on Advanced Sci., Eng. and Inform. Technology 9, 1(2019), 81–91.

[59]

Milagros Miceli, Martin Schuessler, and Tianling Yang. 2020. Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision. arXiv preprint arXiv:2007.14886(2020).

[60]

Annemarie Mol, Ingunn Moser, and Jeannette Pols. 2015. Care in practice: On tinkering in clinics, homes and farms. Vol. 8. transcript Verlag.

[61]

Naja Holten Møller, Claus Bossen, Kathleen H Pine, Trine Rask Nielsen, and Gina Neff. 2020. Who does the work of data?Interactions 27, 3 (2020), 52–55.

[62]

Michael Muller, Ingrid Lange, Dakuo Wang, David Piorkowski, Jason Tsay, Q Vera Liao, Casey Dugan, and Thomas Erickson. 2019. How data science workers work with data: Discovery, capture, curation, design, creation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–15.

Digital Library

[63]

Michael Muller, Christine T Wolf, Josh Andres, Michael Desmond, Narendra Nath Joshi, Zahra Ashktorab, Aabhas Sharma, Kristina Brimijoin, Qian Pan, Evelyn Duesterwald, 2021. Designing Ground Truth and the Social Life of Labels. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16.

Digital Library

[64]

Andrew B Neang, Will Sutherland, Michael W Beach, and Charlotte P Lee. 2021. Data Integration as Coordination: The Articulation of Data Work in an Ocean Science Collaboration. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3(2021), 1–25.

Digital Library

[65]

Gina Neff, Anissa Tanweer, Brittany Fiore-Gartland, and Laura Osburn. 2017. Critique and contribute: A practice-based framework for improving critical data studies and data science. Big data 5, 2 (2017), 85–97.

[66]

Siddharth Nishtala, Harshavardhan Kamarthi, Divy Thakkar, Dhyanesh Narayanan, Anirudh Grama, Ramesh Padmanabhan, Neha Madhiwalla, Suresh Chaudhary, Balaraman Ravindra, and Milind Tambe. 2020. Missed calls, Automated Calls and Health Support: Using AI to improve maternal health outcomes by increasing program engagement. arXiv preprint arXiv:2006.07590(2020).

[67]

Eirini Ntoutsi, Pavlos Fafalios, Ujwal Gadiraju, Vasileios Iosifidis, Wolfgang Nejdl, Maria-Esther Vidal, Salvatore Ruggieri, Franco Turini, Symeon Papadopoulos, Emmanouil Krasanakis, 2020. Bias in data-driven artificial intelligence systems—An introductory survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 10, 3(2020), e1356.

[68]

Chinasa T Okolo, Srujana Kamath, Nicola Dell, and Aditya Vashistha. 2021. “It cannot do all of my work”: Community Health Worker Perceptions of AI-Enabled Mobile Health Applications in Rural India. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–20.

Digital Library

[69]

Cathy O’neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books.

Digital Library

[70]

Joyojeet Pal, Anjuli Dasika, Ahmad Hasan, Jackie Wolf, Nick Reid, Vaishnav Kameswaran, Purva Yardi, Allyson Mackay, Abram Wagner, Bhramar Mukherjee, 2017. Changing data practices for community health workers: Introducing digital data collection in West Bengal, India. In Proceedings of the Ninth International Conference on Information and Communication Technologies and Development. ACM, 17.

Digital Library

[71]

Ankur Pandey, Inshita Mutreja, Saru Brar, and Pushpendra Singh. 2020. Exploring Automated Q&A Support System for Maternal and Child Health in Rural India. In Proceedings of the 3rd ACM SIGCAS Conference on Computing and Sustainable Societies. 349–350.

Digital Library

[72]

Elisavet Parselia, Charalampos Kontoes, Alexia Tsouni, Christos Hadjichristodoulou, Ioannis Kioutsioukis, Gkikas Magiorkinis, and Nikolaos I Stilianakis. 2019. Satellite Earth Observation Data in Epidemiological Modeling of Malaria, Dengue and West Nile Virus: A Scoping Review. Remote Sensing 11, 16 (2019), 1862.

[73]

Samir Passi and Steven Jackson. 2017. Data vision: Learning to see through algorithmic abstraction. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 2436–2447.

Digital Library

[74]

Samir Passi and Steven J Jackson. 2018. Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects. 2 (2018), 136: 1–136: 28. Issue CSCW.

Digital Library

[75]

Amandalynne Paullada, Inioluwa Deborah Raji, Emily M Bender, Emily Denton, and Alex Hanna. 2020. Data and its (dis) contents: A survey of dataset development and use in machine learning research. arXiv preprint arXiv:2012.05345(2020).

[76]

Fahad Pervaiz, Aditya Vashistha, and Richard Anderson. 2019. Examining the challenges in development data pipeline. In Proceedings of the 2nd ACM SIGCAS Conference on Computing and Sustainable Societies. 13–21.

Digital Library

[77]

Kathleen H Pine and Max Liboiron. 2015. The politics of measurement and action. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 3147–3156.

Digital Library

[78]

David Piorkowski, Soya Park, April Yi Wang, Dakuo Wang, Michael Muller, and Felix Portnoy. 2021. How ai developers overcome communication challenges in a multidisciplinary team: A case study. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–25.

Digital Library

[79]

Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 33–44.

Digital Library

[80]

Katelyn J. Rittenhouse, Bellington Vwalika, Alexander Keil, Jennifer Winston, Marie Stoner, Joan T. Price, Monica Kapasa, Mulaya Mubambe, Vanilla Banda, Whyson Muunga, and Jeffrey S. A. Stringer. 2019. Improving preterm newborn identification in low-resource settings with machine learning. PLOS ONE 14, 2 (Feb. 2019), e0198919. https://doi.org/10.1371/journal.pone.0198919

[81]

Minna Ruckenstein and Natasha Dow Schüll. 2017. The datafication of health. Annual Review of Anthropology 46 (2017), 261–278.

[82]

Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora Aroyo. 2021. ”Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.

[83]

Nithya Sambasivan and Rajesh Veeraraghavan. 2022. From Field Experts to Data Collectors: Deskilling of Domain Expertise in AI Development. In CHI 2022.

[84]

Morgan Klaus Scheuerman, Emily Denton, and Alex Hanna. 2021. Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development. arXiv preprint arXiv:2108.04308(2021).

[85]

I Seidman. 2006. A guide for researchers in education and the social sciences.

[86]

Mark Sendak, Madeleine Clare Elish, Michael Gao, Joseph Futoma, William Ratliff, Marshall Nichols, Armando Bedoya, Suresh Balu, and Cara O’Brien. 2020. ” The human body is a black box” supporting clinical decision-making with deep learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 99–109.

Digital Library

[87]

Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D Sculley. 2017. No classification without representation: Assessing geodiversity issues in open data sets for the developing world. arXiv preprint arXiv:1711.08536(2017).

[88]

Ranjit Singh. 2009. Study the Imbrication: A Methodological Maxim to follow the multiple lives of data. Lives of Data 56(2009), 51.

[89]

Chris Speed and Deborah Maxwell. 2015. Designing through value constellations. interactions 22, 5 (2015), 38–43.

[90]

Charles Sutton, Timothy Hobson, James Geddes, and Rich Caruana. 2018. Data diff: Interpretable, executable summaries of changes in distributions for data wrangling. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2279–2288.

Digital Library

[91]

Alex S Taylor, Siân Lindley, Tim Regan, David Sweeney, Vasillis Vlachokyriakos, Lillie Grainger, and Jessica Lingel. 2015. Data-in-place: Thinking through the relations between data and community. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2863–2872.

Digital Library

[92]

Divy Thakkar, Neha Kumar, and Nithya Sambasivan. 2020. Towards an AI-powered future that works for vocational workers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[93]

David R Thomas. 2006. A general inductive approach for analyzing qualitative evaluation data. American journal of evaluation 27, 2 (2006), 237–246.

[94]

Kenneth Thomsen, Lars Iversen, Therese Louise Titlestad, and Ole Winther. 2020. Systematic review of machine learning for diagnosis and prognosis in dermatology. Journal of Dermatological Treatment 31, 5 (2020), 496–510.

[95]

Edward Velasco, Tumacha Agheneza, Kerstin Denecke, Goeran Kirchner, and Tim Eckmanns. 2014. Social media and internet-based data in global systems for public health surveillance: a systematic review. The Milbank Quarterly 92, 1 (2014), 7–33.

[96]

Maranke Wieringa. 2020. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 1–18.

Digital Library

[97]

Ka Wong, Praveen Paritosh, and Lora Aroyo. 2021. Cross-replication Reliability–An Empirical Approach to Interpreting Inter-rater Reliability. arXiv preprint arXiv:2106.07393(2021).

[98]

Deepika Yadav, Prerna Malik, Kirti Dabas, and Pushpendra Singh. 2019. Feedpal: Understanding Opportunities for Chatbots in Breastfeeding Education of Women in India. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 170.

Digital Library

[99]

Qian Yang, Aaron Steinfeld, Carolyn Rosé, and John Zimmerman. 2020. Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design. In Proceedings of the 2020 chi conference on human factors in computing systems. 1–13.

Digital Library

[100]

Qian Yang, Aaron Steinfeld, and John Zimmerman. 2019. Unremarkable AI: Fitting intelligent decision support into critical, clinical decision-making processes. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.

Digital Library

[101]

Lindsay E Young, Jerome Mayaud, Sze-Chuan Suen, Milind Tambe, and Eric Rice. 2020. Modeling the dynamism of HIV information diffusion in multiplex networks of homeless youth. Social Networks 63(2020), 112–121.

[102]

Ellen Zegura, Carl DiSalvo, and Amanda Meng. 2018. Care and the practice of data science for social good. In Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies. 1–9.

Digital Library

[103]

Amy X Zhang, Michael Muller, and Dakuo Wang. 2020. How do data science workers collaborate? roles, workflows, and tools. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1(2020), 1–23.

Digital Library

[104]

James Zou and Londa Schiebinger. 2018. AI can be sexist and racist—it’s time to make it fair.

Cited By

Balagopalan ABaldini ICeli LGichoya JMcCoy LNaumann TShalit Uvan der Schaar MWagstaff K(2024)Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impactPLOS Digital Health10.1371/journal.pdig.00004743:4(e0000474)Online publication date: 15-Apr-2024
https://doi.org/10.1371/journal.pdig.0000474
Lin HKarusala NOkolo CD'Ignazio CGajos K(2024)"Come to us first": Centering Community Organizations in Artificial Intelligence for Social Good PartnershipsProceedings of the ACM on Human-Computer Interaction10.1145/36870098:CSCW2(1-28)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687009
Huber LSingh ADombrowski LGuha SHardy JHolten Møller NFarzan RLópez CCardoso Llach DQuercia DMustafa MNiu SWong-Villacrés M(2024)Datafication Dilemmas: Data Governance in the Public InterestCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3689137(114-116)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1145/3678884.3689137
Show More Cited By

Index Terms

When is Machine Learning Data Good?: Valuing in Public Health Datafication
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Engaging Solidarity in Data Collection Practices for Community Health

Researchers and practitioners engaged in global health are increasingly turning to data-driven approaches. However, the institutionalized roles and responsibilities---as well as motivations and workflows---of those involved in data collection are not ...
Inheriting Discrimination: Datafication Encounters of Marginalized Workers
ICTD '22: Proceedings of the 2022 International Conference on Information and Communication Technologies and Development

Grassroots workers are increasingly subjected to data-driven systems worldwide. While there has been increasing attention to processes of datafication in state sponsored welfare programs, not much attention has been focused on everyday workplace of the ...
Evaluating public health uses of health information exchange

Health information exchange (HIE) initiatives are in various stages of development across the United States. They aim to bring previously unavailable clinical data from patients' disparate health records, which may be spread over multiple provider and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
2,204
Total Downloads

Downloads (Last 12 months)783
Downloads (Last 6 weeks)66

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Balagopalan ABaldini ICeli LGichoya JMcCoy LNaumann TShalit Uvan der Schaar MWagstaff K(2024)Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impactPLOS Digital Health10.1371/journal.pdig.00004743:4(e0000474)Online publication date: 15-Apr-2024
https://doi.org/10.1371/journal.pdig.0000474
Lin HKarusala NOkolo CD'Ignazio CGajos K(2024)"Come to us first": Centering Community Organizations in Artificial Intelligence for Social Good PartnershipsProceedings of the ACM on Human-Computer Interaction10.1145/36870098:CSCW2(1-28)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687009
Huber LSingh ADombrowski LGuha SHardy JHolten Møller NFarzan RLópez CCardoso Llach DQuercia DMustafa MNiu SWong-Villacrés M(2024)Datafication Dilemmas: Data Governance in the Public InterestCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3689137(114-116)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1145/3678884.3689137
Karusala NUpadhyay SVeeraraghavan RGajos K(2024)Understanding Contestability on the Margins: Implications for the Design of Algorithmic Decision-making in Public ServicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641898(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641898
Bertelsen PBossen CKnudsen CPedersen A(2024)Data work and practices in healthcare: A scoping reviewInternational Journal of Medical Informatics10.1016/j.ijmedinf.2024.105348184(105348)Online publication date: Apr-2024
https://doi.org/10.1016/j.ijmedinf.2024.105348
Pedersen ABossen C(2024)Cultivating Data Practices Across Boundaries: How Organizations Become Data-drivenComputer Supported Cooperative Work (CSCW)10.1007/s10606-024-09489-833:4(1177-1221)Online publication date: 28-Feb-2024
https://doi.org/10.1007/s10606-024-09489-8
Sharma VIsmail A(2023)Lessons from Ph.D. Fieldwork: A Conversation with Dr. Azra IsmailXRDS: Crossroads, The ACM Magazine for Students10.1145/361168830:1(70-73)Online publication date: 20-Oct-2023
https://dl.acm.org/doi/10.1145/3611688
Ismail AThakkar DMadhiwalla NKumar N(2023)Public Health Calls for/with AI: An Ethnographic PerspectiveProceedings of the ACM on Human-Computer Interaction10.1145/36102037:CSCW2(1-26)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610203
Pant DTalukder DSeth APant DSingh RDua BPandey RMaruthi SJohri MArora C(2023)Robust OCR Pipeline for Automated Digitization of Mother and Child Protection Cards in IndiaACM Journal on Computing and Sustainable Societies10.1145/36081141:1(1-24)Online publication date: 22-Sep-2023
https://dl.acm.org/doi/10.1145/3608114
Priestley MO’donnell FSimperl E(2023)A Survey of Data Quality Requirements That Matter in ML Development PipelinesJournal of Data and Information Quality10.1145/359261615:2(1-39)Online publication date: 22-Jun-2023
https://dl.acm.org/doi/10.1145/3592616
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten