research-article

Speaking from Experience: Trans/Non-Binary Requirements for Voice-Activated AI

Authors:

Corinne CathAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 5, Issue CSCW1

Article No.: 132, Pages 1 - 27

https://doi.org/10.1145/3449206

Published: 22 April 2021 Publication History

Abstract

Voice-Activated Artificial Intelligence (VAI) is increasingly ubiquitous, whether appearing as context-specific conversational assistants or more personalised and generalised personal assistants such as Alexa or Siri. CSCW and other fields have regularly studied the (positive and negative) social consequences of VAI design and deployment. One particular focus has been questions of gender, and the implications that the (often-feminine) gendering of VAIs has for societal norms and user experiences. Studies into this have largely elided transgender (trans) existences; the few exceptions to this operate largely from an external and predetermined idea of trans and/or non-binary user needs, centered on representation. In this study, we undertook a series of qualitative interviews with trans and/or non-binary users of VAIs to explore their experiences and needs. Our results show that these needs are far more than simply improvements to representation, and that users raise substantial concerns around the underlying framing of gender by even well-intentioned developers, the privacy and safety implications of ubiquitous VAI, and the motivations of the vast for-profit companies that deploy much of this technology. We provide both immediate recommendations for designers and researchers seeking to create trans-inclusive VAIs, and wider, critical proposals for how we as researchers go about assessing technological systems and appropriate points of intervention.

Supplementary Material

ZIP File (v5cscw132aux.zip)

Download
152.33 KB

References

[1]

Sofia Aboim. 2020. Fragmented Recognition: Gender Identity between Moral and Legal Spheres. Social Politics: International Studies in Gender, State & Society (2020).

[2]

Alex A Ahmed. 2018. Trans Competent Interaction Design: A Qualitative Study on Voice, Identity, and Technology. Interacting with Computers 30, 1 (2018), 53--71.

[3]

Alex A Ahmed. 2019. Bridging Social Critique and Design: Building a Health Informatics Tool for Transgender Voice. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1--4.

Digital Library

[4]

James F Allen, Donna K Byron, Myroslava Dzikovska, George Ferguson, Lucian Galescu, and Amanda Stent. 2001. Toward conversational human-computer interaction. AI magazine 22, 4 (2001), 27--27.

Digital Library

[5]

Meryl Alper. 2017. Talking Like a "Princess": What Speaking Machines Say About Human Biases. The New Inquiry (2017).

[6]

H Russell Bernard, Amber Wutich, and Gery W Ryan. 2016. Analyzing qualitative data: Systematic approaches. SAGE publications.

[7]

Geoffrey C Bowker and Susan Leigh Star. 2000. Sorting things out: Classification and its consequences. MIT press.

Digital Library

[8]

Danah Michele Boyd. 2008. Taken out of context: American teen sociality in networked publics. Ph.D. Dissertation. University of California, Berkeley.

[9]

Michael Braun, Anja Mainz, Ronee Chadowitz, Bastian Pfleging, and Florian Alt. 2019. At your service: Designing voice assistant personalities to improve automotive user interfaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--11.

Digital Library

[10]

Samantha Breslin and Bimlesh Wadhwa. 2014. Exploring nuanced gender perspectives within the HCI community. In Proceedings of the india hci 2014 conference on human computer interaction. 45--54.

Digital Library

[11]

Julia Cambre and Chinmay Kulkarni. 2019. One Voice Fits All? Social Implications and Research Challenges of Designing Voices for Smart Devices. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1--19.

[12]

Julie Carpenter. 2019. Why project Q is more than the world's first nonbinary voice for technology. interactions 26, 6 (2019), 56--59.

[13]

Kathy Charmaz. 2014. Constructing grounded theory. sage.

[14]

Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, et al. 2019. What makes a good conversation? challenges in designing truly conversational agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--12.

Digital Library

[15]

Cynthia Cockburn. 1992. The circuit of technology: gender, identity and power. Consuming technologies: Media and information in domestic spaces (1992), 33--42.

[16]

Robert Dale, Hermann Moisl, and Harold Somers. 2000. Handbook of natural language processing. CRC Press.

[17]

Helana Darwin. 2020. Challenging the Cisgender/Transgender Binary: Nonbinary People and the Transgender Label. Gender & Society (2020), 0891243220912256.

[18]

Emmanuel David. 2017. Capital T: Trans visibility, corporate capitalism, and commodity culture. Transgender Studies Quarterly 4 (2017), 28--44. Issue 1.

[19]

Dylan Amy Davis. 2017. The normativity of recognition: Non-binary gender markers in Australian law and policy. Gender panic, gender policy 24 (2017), 227--250.

[20]

Paul Dourish. 2003. The appropriation of interactive technologies: Some lessons from placeless documents. Computer Supported Cooperative Work (CSCW) 12, 4 (2003), 465--490.

Digital Library

[21]

Carl Benedikt Frey and Michael A Osborne. 2017. The future of employment: How susceptible are jobs to computerisation? Technological forecasting and social change 114 (2017), 254--280.

[22]

Sarah Gold. 2019. Designing Permissions Worth Trusting. https://www.projectsbyif.com/blog/how-do-you-designpermissions- worth-trusting/

[23]

Tim Guetterman. 2015. Descriptions of sampling practices within five approaches to qualitative research in education and the health sciences. (2015).

[24]

Oliver L. Haimson, Dykee Gorrell, Denny L. Starks, and Zu Weinger. 2020. Designing Trans Technology: Defining Challenges and Envisioning Community-Centered Solutions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https: //doi.org/10.1145/3313831.3376669

Digital Library

[25]

Christopher B. Devichand M. Tunstall-Pedoe W. Higham E. Hamed, H. and V Tieser. 2019. Smart Speakers and Voice Assistants. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/831180/ Snapshot_Paper_-_Smart_Speakers_and_Voice_Assistants.pdf

[26]

Charles Hannon. 2016. Gender and status in voice user interfaces. interactions 23, 3 (2016), 34--37.

[27]

INCITE! Women, Gender Non-Conforming, and Trans people of Color Against Violence. 2007. The revolution will not be funded: Beyond the non-profit industrial complex. South End Press.

[28]

Gopinaath Kannabiran. 2017. " Embodied Wellbeing": A Re-Imagination of Sustainability and Desire in HCI. Ph.D. Dissertation. Indiana University.

[29]

Os Keyes. 2018. The misgendering machines: Trans/HCI implications of automatic gender recognition. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1--22.

Digital Library

[30]

Os Keyes. 2019. Counting the Countless: Why data science is a profound threat for queer people. Real Life 2 (2019).

[31]

Os Keyes, Josephine Hoy, and Margaret Drouhard. 2019. Human-Computer Insurrection: Notes on an Anarchist HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[32]

Chandra Khatri, Anu Venkatesh, Behnam Hedayatnia, Raefer Gabriel, Ashwin Ram, and Rohit Prasad. 2018. Alexa Prize-State of the Art in Conversational AI. AI Magazine 39, 3 (2018), 40--55.

Digital Library

[33]

Danya Lagos. 2019. Hearing Gender: Voice-Based Gender Classification Processes and Transgender Health Inequality. American Sociological Review 84, 5 (2019), 801--827.

[34]

Brian N Larson. 2017. Gender as a Variable in Natural-Language Processing: Ethical Considerations. EACL 2017 (2017), 1.

[35]

Ada Lerner, Helen Yuxun He, Anna Kawakami, Silvia Catherine Zeamer, and Roberto Hoyle. 2020. Privacy and Activism in the Transgender Community. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[36]

Q Vera Liao, Yi-Chia Wang, Timothy Bickmore, Pascale Fung, Jonathan Grudin, Zhou Yu, and Michelle Zhou. 2019. Human-Agent Communication: Connecting Research and Development in HCI and AI. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing. 122--126.

Digital Library

[37]

Ida Linander, Isabel Goicolea, Erika Alm, Anne Hammarström, and Lisa Harryson. 2019. (Un) safe spaces, affective labour and perceived health among people with trans experiences living in Sweden. Culture, health & sexuality 21, 8 (2019), 914--928.

[38]

Irene Lopatovska, Katrina Rink, Ian Knight, Kieran Raines, Kevin Cosenza, Harriet Williams, Perachya Sorsche, David Hirsch, Qi Li, and Adrianna Martinez. 2019. Talk to me: Exploring user interactions with the Amazon Alexa. Journal of Librarianship and Information Science 51, 4 (2019), 984--997.

[39]

Rickke Mananzala and Dean Spade. 2008. The nonprofit industrial complex and trans resistance. Sexuality Research & Social Policy 5, 1 (2008), 53.

[40]

Patchen Markell. 2009. Bound by recognition. Princeton University Press.

[41]

Moira McGregor and John C Tang. 2017. More to meetings: challenges in using speech-based technology to support meetings. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 2208--2220.

Digital Library

[42]

Donald McMillan, Barry Brown, Ikkaku Kawaguchi, Razan Jaber, Jordi Solsona Belenguer, and Hideaki Kuzuoka. 2019. Designing with Gaze: Tama--a Gaze Activated Smart-Speaker. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1--26.

Digital Library

[43]

Andrew McStay. 2018. Emotional AI: The rise of empathic media. Sage.

[44]

D. Mehta and A-K. Hamke. [n.d.]. In-depth: Artificial Intelligence 2019. https://www.statista.com/study/50485/artificialintelligence/

[45]

Oussama Metatla, Alison Oldfield, Taimur Ahmed, Antonis Vafeas, and Sunny Miglani. 2019. Voice user interfaces in schools: Co-designing for inclusion with visually-impaired and sighted pupils. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--15.

Digital Library

[46]

Matthew B Miles, A Michael Huberman, and Johnny Saldana. 2014. Qualitative Data Analysis: A Methods Sourcebook. SAGE Publications Ltd (CA) (2014).

[47]

J. Moar. [n.d.]. Digital Assistants of Tomorrow. https://www.juniperresearch.com/document-library/white-papers/heysiri- how-will-you-make-money

[48]

Clifford Ivar Nass and Scott Brave. 2005. Wired for speech: How voice activates and advances the human-computer relationship. MIT press Cambridge, MA.

Digital Library

[49]

Helen Nissenbaum. 2004. Privacy as contextual integrity. Wash. L. Rev. 79 (2004), 119.

[50]

Larry Nuttbrock, Walter Bockting, Andrew Rosenblum, Sel Hwahng, Mona Mason, Monica Macri, and Jeffrey Becker. 2014. Gender abuse and major depression among transgender women: a prospective study of vulnerability and resilience. American Journal of Public Health 104, 11 (2014), 2191--2198.

[51]

Christi Olson and Kelli Kemery. 2019. Voice Report. https://advertiseonbing-blob.azureedge.net/blob/bingads/media/ insight/whitepapers/2019/04%20apr/voice-report/bingads_2019_voicereport.pdf

[52]

Lawrence A Palinkas, Sarah M Horwitz, Carla A Green, Jennifer P Wisdom, Naihua Duan, and Kimberly Hoagwood. 2015. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and policy in mental health and mental health services research 42, 5 (2015), 533--544.

[53]

Michael Quinn Patton. 2002. Qualitative evaluation and research methods. SAGE Publications, inc.

[54]

Martin Porcheron, Joel E Fischer, Moira McGregor, Barry Brown, Ewa Luger, Heloisa Candello, and Kenton O'Hara. 2017. Talking with conversational agents in collaborative action. In companion of the 2017 ACM conference on computer supported cooperative work and social computing. 431--436.

[55]

Martin Porcheron, Joel E Fischer, and Sarah Sharples. 2017. "Do Animals Have Accents?" Talking with Agents in Multi-Party Conversation. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing. 207--219.

[56]

Alisha Pradhan, Leah Findlater, and Amanda Lazar. 2019. "Phantom Friend" or" Just a Box with Information" Personification and Ontological Categorization of Smart Speaker-based Voice Assistants by Older Adults. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1--21.

[57]

Jennifer A Rode. 2011. A theoretical agenda for feminist HCI. Interacting with Computers 23, 5 (2011), 393--400.

Digital Library

[58]

Morgan Klaus Scheuerman, Stacy M Branham, and Foad Hamidi. 2018. Safe spaces and safe places: Unpacking technology-mediated experiences of safety and harm with transgender people. Proceedings of the ACM on Human- Computer Interaction 2, CSCW (2018), 1--27.

Digital Library

[59]

Zoe Schiffer. 2020. This girls-only app uses AI to screen a user's gender - what could go wrong? https://www.theverge. com/2020/2/7/21128236/gender-app-giggle-women-ai-screen-trans-social

[60]

Kristen Schilt and Laurel Westbrook. 2009. Doing Gender, Doing Heteronormativity: "Gender Normals," Transgender People, and the Social Maintenance of Heterosexuality. Gender & society 23, 4 (2009), 440--464.

[61]

Brian Schram. 2019. Accidental Orientations: Rethinking Queerness in Archival Times. Surveillance & Society 17, 5 (2019), 602--617.

[62]

Eve Shapiro. 2015. Gender circuits: Bodies and identities in a technological age. Routledge.

[63]

Dean Spade. 2015. Normal life: Administrative violence, critical trans politics, and the limits of law. Duke University Press.

[64]

Susan Leigh Star and Geoffrey C Bowker. 2007. Enacting silence: Residual categories as a challenge for ethics, information systems, and communication. Ethics and Information Technology 9, 4 (2007), 273--280.

Digital Library

[65]

Denny L Starks, Tawanna Dillahunt, and Oliver L Haimson. 2019. Designing Technology to Support Safety for Transgender Women & Non-Binary People of Color. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion. 289--294.

Digital Library

[66]

Anselm Strauss and Juliet Corbin. 1990. Basics of qualitative research. Sage publications.

[67]

Maurice E Stucke and Ariel Ezrachi. 2017. How digital assistants can harm our economy, privacy, and democracy. Berkeley Tech. LJ 32 (2017), 1239.

[68]

Siva Vaidhyanathan. 2011. The googlization of us: Universal surveillance and infrastructural imperialism. Vol. 82. 82--114.

[69]

Evan Vipond. 2015. Resisting Transnormativity: challenging the medicalization and regulation of trans bodies. Theory in Action 8, 2 (2015).

[70]

MichaelWesch. 2009. Youtube and you: Experiences of self-awareness in the context collapse of the recording webcam. explorations in media ecology 8, 2 (2009), 19--34.

[71]

Mark West, Rebecca Kraut, and Han Ei Chew. 2019. I'd blush if I could: closing gender divides in digital skills through education. (2019).

[72]

Carla Willig. 2013. Introducing qualitative research in psychology. McGraw-hill education (UK).

[73]

Heather Suzanne Woods. 2018. Asking more of Siri and Alexa: feminine persona in service of surveillance capitalism. Critical Studies in Media Communication 35, 4 (2018), 334--349.

[74]

Meredith GF Worthen. 2016. Hetero-cis--normativity and the gendering of transphobia. International Journal of Transgenderism 17, 1 (2016), 31--57.

Cited By

Kausar AKausar AKulkarni SBhosale PDas S(2025)Gender Bias in AIDimensions of Diversity, Equity, Inclusion, and Belonging in Business10.4018/979-8-3693-3876-6.ch004(87-106)Online publication date: 10-Jan-2025
https://doi.org/10.4018/979-8-3693-3876-6.ch004
Haimson ODeGuia ASaber RBrewster K(2024)Extended Reality Trans Technologies: Bridging Digital and Physical Worlds to Support Transgender PeopleProceedings of the ACM on Human-Computer Interaction10.1145/36869728:CSCW2(1-27)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686972
Wenzel KKaufman G(2024)Designing for Harm Reduction: Communication Repair for Multicultural Users' Voice InteractionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642900(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642900
Show More Cited By

Index Terms

Speaking from Experience: Trans/Non-Binary Requirements for Voice-Activated AI

Recommendations

Patching Gender: Non-binary Utopias in HCI
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Non-binary people are rarely considered by technologies or technologists, and often subsumed under binary trans experiences on the rare occasions when we are discussed. In this paper we share our own experiences and explore potential alternatives - ...
"Genderfluid" or "Attack Helicopter": Responsible HCI Research Practice with Non-binary Gender Variation in Online Communities
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

As non-binary genders become increasingly prevalent, researchers face decisions in how to collect, analyze and interpret research participants' genders. We present two case studies on surveys with thousands of respondents, of which hundreds reported ...
The role of theory in gender and information systems research

In response to claims emanating from recent assessments of the status of gender and IS research about insufficient theorizing of gender, a critical literature analysis of research papers on the topic of gender and IS that appeared in information systems ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 5, Issue CSCW1

CSCW

April 2021

5016 pages

EISSN:2573-0142

DOI:10.1145/3460939

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2021

Published in PACMHCI Volume 5, Issue CSCW1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
611
Total Downloads

Downloads (Last 12 months)207
Downloads (Last 6 weeks)26

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kausar AKausar AKulkarni SBhosale PDas S(2025)Gender Bias in AIDimensions of Diversity, Equity, Inclusion, and Belonging in Business10.4018/979-8-3693-3876-6.ch004(87-106)Online publication date: 10-Jan-2025
https://doi.org/10.4018/979-8-3693-3876-6.ch004
Haimson ODeGuia ASaber RBrewster K(2024)Extended Reality Trans Technologies: Bridging Digital and Physical Worlds to Support Transgender PeopleProceedings of the ACM on Human-Computer Interaction10.1145/36869728:CSCW2(1-27)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686972
Wenzel KKaufman G(2024)Designing for Harm Reduction: Communication Repair for Multicultural Users' Voice InteractionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642900(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642900
Taylor JSimpson ETran ABrubaker JFox SZhu H(2024)Cruising Queer HCI on the DL: A Literature Review of LGBTQ+ People in HCIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642494(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642494
Habiba UHaug MBogner JWagner S(2024)How mature is requirements engineering for AI-based systems? A systematic mapping study on practices, challenges, and future research directionsRequirements Engineering10.1007/s00766-024-00432-329:4(567-600)Online publication date: 23-Oct-2024
https://doi.org/10.1007/s00766-024-00432-3
Fetterolf EHertog E(2023)It’s not her fault: Trust through anthropomorphism among young adult Amazon Alexa usersConvergence: The International Journal of Research into New Media Technologies10.1177/13548565231200337Online publication date: 9-Oct-2023
https://doi.org/10.1177/13548565231200337
Seymour WZhan XCoté MSuch J(2023)A Systematic Review of Ethical Concerns with Voice AssistantsProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3600211.3604679(131-145)Online publication date: 8-Aug-2023
https://dl.acm.org/doi/10.1145/3600211.3604679
Shelby RRismani SHenne KMoon ARostamzadeh NNicholas PYilla-Akbari NGallegos JSmart AGarcia EVirk G(2023)Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm ReductionProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3600211.3604673(723-741)Online publication date: 8-Aug-2023
https://dl.acm.org/doi/10.1145/3600211.3604673
Liliequist ETubella ADanielsson KCocq C(2023)Beyond the Binary --- Queering AI for an Inclusive FutureInteractions10.1145/359014130:3(31-33)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1145/3590141
Seymour WZhan XCote MSuch J(2023)Who are CUIs Really For? Representation and Accessibility in the Conversational User Interface LiteratureProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3603760(1-5)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3603760
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents