-
Experiences of Censorship on TikTok Across Marginalised Identities
Authors:
Eddie L. Ungless,
Nina Markl,
Björn Ross
Abstract:
TikTok has seen exponential growth as a platform, fuelled by the success of its proprietary recommender algorithm which serves tailored content to every user - though not without controversy. Users complain of their content being unfairly suppressed by ''the algorithm'', particularly users with marginalised identities such as LGBTQ+ users. Together with content removal, this suppression acts to ce…
▽ More
TikTok has seen exponential growth as a platform, fuelled by the success of its proprietary recommender algorithm which serves tailored content to every user - though not without controversy. Users complain of their content being unfairly suppressed by ''the algorithm'', particularly users with marginalised identities such as LGBTQ+ users. Together with content removal, this suppression acts to censor what is shared on the platform. Journalists have revealed biases in automatic censorship, as well as human moderation. We investigate experiences of censorship on TikTok, across users marginalised by their gender, LGBTQ+ identity, disability or ethnicity. We survey 627 UK-based TikTok users and find that marginalised users often feel they are subject to censorship for content that does not violate community guidelines. We highlight many avenues for future research into censorship on TikTok, with a focus on users' folk theories, which greatly shape their experiences of the platform.
△ Less
Submitted 19 July, 2024;
originally announced July 2024.
-
Just Because We Camp, Doesn't Mean We Should: The Ethics of Modelling Queer Voices
Authors:
Atli Sigurgeirsson,
Eddie L. Ungless
Abstract:
Modern voice cloning models claim to be able to capture a diverse range of voices. We test the ability of a typical pipeline to capture the style known colloquially as "gay voice" and notice a homogenisation effect: synthesised speech is rated as sounding significantly "less gay" (by LGBTQ+ participants) than its corresponding ground-truth for speakers with "gay voice", but ratings actually increa…
▽ More
Modern voice cloning models claim to be able to capture a diverse range of voices. We test the ability of a typical pipeline to capture the style known colloquially as "gay voice" and notice a homogenisation effect: synthesised speech is rated as sounding significantly "less gay" (by LGBTQ+ participants) than its corresponding ground-truth for speakers with "gay voice", but ratings actually increase for control speakers. Loss of "gay voice" has implications for accessibility. We also find that for speakers with "gay voice", loss of "gay voice" corresponds to lower similarity ratings.
However, we caution that improving the ability of such models to synthesise ``gay voice'' comes with a great number of risks. We use this pipeline as a starting point for a discussion on the ethics of modelling queer voices more broadly. Collecting "clean" queer data has safety and fairness ramifications, and the resulting technology may cause harms from mockery to death.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
Typology of Risks of Generative Text-to-Image Models
Authors:
Charlotte Bird,
Eddie L. Ungless,
Atoosa Kasirzadeh
Abstract:
This paper investigates the direct risks and harms associated with modern text-to-image generative models, such as DALL-E and Midjourney, through a comprehensive literature review. While these models offer unprecedented capabilities for generating images, their development and use introduce new types of risk that require careful consideration. Our review reveals significant knowledge gaps concerni…
▽ More
This paper investigates the direct risks and harms associated with modern text-to-image generative models, such as DALL-E and Midjourney, through a comprehensive literature review. While these models offer unprecedented capabilities for generating images, their development and use introduce new types of risk that require careful consideration. Our review reveals significant knowledge gaps concerning the understanding and treatment of these risks despite some already being addressed. We offer a taxonomy of risks across six key stakeholder groups, inclusive of unexplored issues, and suggest future research directions. We identify 22 distinct risk types, spanning issues from data bias to malicious use. The investigation presented here is intended to enhance the ongoing discourse on responsible model development and deployment. By highlighting previously overlooked risks and gaps, it aims to shape subsequent research and governance initiatives, guiding them toward the responsible, secure, and ethically conscious evolution of text-to-image models.
△ Less
Submitted 8 July, 2023;
originally announced July 2023.
-
Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models
Authors:
Eddie L. Ungless,
Björn Ross,
Anne Lauscher
Abstract:
Cutting-edge image generation has been praised for producing high-quality images, suggesting a ubiquitous future in a variety of applications. However, initial studies have pointed to the potential for harm due to predictive bias, reflecting and potentially reinforcing cultural stereotypes. In this work, we are the first to investigate how multimodal models handle diverse gender identities. Concre…
▽ More
Cutting-edge image generation has been praised for producing high-quality images, suggesting a ubiquitous future in a variety of applications. However, initial studies have pointed to the potential for harm due to predictive bias, reflecting and potentially reinforcing cultural stereotypes. In this work, we are the first to investigate how multimodal models handle diverse gender identities. Concretely, we conduct a thorough analysis in which we compare the output of three image generation models for prompts containing cisgender vs. non-cisgender identity terms. Our findings demonstrate that certain non-cisgender identities are consistently (mis)represented as less human, more stereotyped and more sexualised. We complement our experimental analysis with (a)~a survey among non-cisgender individuals and (b) a series of interviews, to establish which harms affected individuals anticipate, and how they would like to be represented. We find respondents are particularly concerned about misrepresentation, and the potential to drive harmful behaviours and beliefs. Simple heuristics to limit offensive content are widely rejected, and instead respondents call for community involvement, curated training data and the ability to customise. These improvements could pave the way for a future where change is led by the affected community, and technology is used to positively ``[portray] queerness in ways that we haven't even thought of'' rather than reproducing stale, offensive stereotypes.
△ Less
Submitted 26 May, 2023;
originally announced May 2023.
-
A Robust Bias Mitigation Procedure Based on the Stereotype Content Model
Authors:
Eddie L. Ungless,
Amy Rafferty,
Hrichika Nag,
Björn Ross
Abstract:
The Stereotype Content model (SCM) states that we tend to perceive minority groups as cold, incompetent or both. In this paper we adapt existing work to demonstrate that the Stereotype Content model holds for contextualised word embeddings, then use these results to evaluate a fine-tuning process designed to drive a language model away from stereotyped portrayals of minority groups. We find the SC…
▽ More
The Stereotype Content model (SCM) states that we tend to perceive minority groups as cold, incompetent or both. In this paper we adapt existing work to demonstrate that the Stereotype Content model holds for contextualised word embeddings, then use these results to evaluate a fine-tuning process designed to drive a language model away from stereotyped portrayals of minority groups. We find the SCM terms are better able to capture bias than demographic agnostic terms related to pleasantness. Further, we were able to reduce the presence of stereotypes in the model through a simple fine-tuning procedure that required minimal human and computer resources, without harming downstream performance. We present this work as a prototype of a debiasing procedure that aims to remove the need for a priori knowledge of the specifics of bias in the model.
△ Less
Submitted 26 October, 2022;
originally announced October 2022.
