Reporting Non-Consensual Intimate Media: An Audit Study of Deepfakes
0
Sign in to get full access
Overview
- Examines the process of reporting non-consensual intimate media (NCIM), including deepfakes, on social media platforms
- Conducts an audit study to understand the effectiveness of reporting mechanisms for this type of content
- Explores the challenges and limitations of current content moderation practices for NCIM
Plain English Explanation
This paper investigates the process of reporting non-consensual intimate media, which includes fake explicit images and videos (also known as deepfakes), on social media platforms. The researchers conducted an audit study to understand how effective the reporting mechanisms are for this type of abusive online content.
The study reveals the challenges and limitations of current content moderation practices for non-consensual intimate media. The findings suggest that the reporting process can be confusing and ineffective, leaving victims without adequate support to address this form of online abuse.
Technical Explanation
The researchers designed an audit study to evaluate the effectiveness of reporting mechanisms for non-consensual intimate media on social media platforms. They created several test accounts and attempted to report different types of NCIM, including deepfakes, across multiple platforms.
The study analyzed the reporting process, platform responses, and the overall outcomes to understand the challenges and limitations of current content moderation practices. The researchers found that the reporting process was often unclear, with inconsistent responses from platforms and limited support for victims.
The paper also discusses the broader implications of these findings, highlighting the need for more robust and victim-centric approaches to addressing non-consensual intimate media online.
Critical Analysis
The audit study provides valuable insights into the challenges of reporting non-consensual intimate media, but the researchers acknowledge several limitations. The study was limited in scope, focusing on a small number of platforms and test cases, and the findings may not fully represent the experiences of all victims.
Additionally, the paper does not delve into the technical aspects of detecting and removing deepfakes or other forms of NCIM, which could be an area for further research. The researchers also note the need for greater collaboration between platforms, policymakers, and advocacy groups to develop more effective solutions.
Conclusion
This paper highlights the significant challenges faced by victims of non-consensual intimate media when attempting to report and address this type of online abuse. The audit study findings suggest that current content moderation practices are often inadequate, leaving victims without the support they need.
The researchers call for a more comprehensive and victim-centric approach to addressing non-consensual intimate media, including improved reporting mechanisms, better detection and removal capabilities, and increased collaboration among stakeholders. Addressing this issue is crucial to protect the rights and wellbeing of individuals affected by this type of online abuse.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
New!Reporting Non-Consensual Intimate Media: An Audit Study of Deepfakes
Li Qiwei, Shihui Zhang, Andrew Timothy Kasper, Joshua Ashkinaze, Asia A. Eaton, Sarita Schoenebeck, Eric Gilbert
Non-consensual intimate media (NCIM) inflicts significant harm. Currently, victim-survivors can use two mechanisms to report NCIM - as a non-consensual nudity violation or as copyright infringement. We conducted an audit study of takedown speed of NCIM reported to X (formerly Twitter) of both mechanisms. We uploaded 50 AI-generated nude images and reported half under X's non-consensual nudity reporting mechanism and half under its copyright infringement mechanism. The copyright condition resulted in successful image removal within 25 hours for all images (100% removal rate), while non-consensual nudity reports resulted in no image removal for over three weeks (0% removal rate). We stress the need for targeted legislation to regulate NCIM removal online. We also discuss ethical considerations for auditing NCIM on social platforms.
Read more9/19/2024
0
Violation of my body: Perceptions of AI-generated non-consensual (intimate) imagery
Natalie Grace Brigham, Miranda Wei, Tadayoshi Kohno, Elissa M. Redmiles
AI technology has enabled the creation of deepfakes: hyper-realistic synthetic media. We surveyed 315 individuals in the U.S. on their views regarding the hypothetical non-consensual creation of deepfakes depicting them, including deepfakes portraying sexual acts. Respondents indicated strong opposition to creating and, even more so, sharing non-consensually created synthetic content, especially if that content depicts a sexual act. However, seeking out such content appeared more acceptable to some respondents. Attitudes around acceptability varied further based on the hypothetical creator's relationship to the participant, the respondent's gender and their attitudes towards sexual consent. This study provides initial insight into public perspectives of a growing threat and highlights the need for further research to inform social norms as well as ongoing policy conversations and technical developments in generative AI.
Read more6/18/2024
📊
0
The Sociotechnical Stack: Opportunities for Social Computing Research in Non-consensual Intimate Media
Li Qiwei, Allison McDonald, Oliver L. Haimson, Sarita Schoenebeck, Eric Gilbert
Non-consensual intimate media (NCIM) involves sharing intimate content without the depicted person's consent, including revenge porn and sexually explicit deepfakes. While NCIM has received attention in legal, psychological, and communication fields over the past decade, it is not sufficiently addressed in computing scholarship. This paper addresses this gap by linking NCIM harms to the specific technological components that facilitate them. We introduce the sociotechnical stack, a conceptual framework designed to map the technical stack to its corresponding social impacts. The sociotechnical stack allows us to analyze sociotechnical problems like NCIM, and points toward opportunities for computing research. We propose a research roadmap for computing and social computing communities to deter NCIM perpetration and support victim-survivors through building and rebuilding technologies.
Read more8/7/2024
0
Feminist Interaction Techniques: Deterring Non-Consensual Screenshots with Interaction Techniques
Li Qiwei, Francesca Lameiro, Shefali Patel, Cristi-Isaula-Reyes, Eytan Adar, Eric Gilbert, Sarita Schoenebeck
Non-consensual Intimate Media (NCIM) refers to the distribution of sexual or intimate content without consent. NCIM is common and causes significant emotional, financial, and reputational harm. We developed Hands-Off, an interaction technique for messaging applications that deters non-consensual screenshots. Hands-Off requires recipients to perform a hand gesture in the air, above the device, to unlock media -- which makes simultaneous screenshotting difficult. A lab study shows that Hands-Off gestures are easy to perform and reduce non-consensual screenshots by 67 percent. We conclude by generalizing this approach and introduce the idea of Feminist Interaction Techniques (FIT), interaction techniques that encode feminist values and speak to societal problems, and reflect on FIT's opportunities and limitations.
Read more8/7/2024