I Searched for a Religious Song in Amharic and Got Sexual Content Instead: Investigating Online Harm in Low-Resourced Languages on YouTube

Overview

• This paper investigates online harm in low-resourced languages on the YouTube platform.
• The researchers focus on the Amharic language and explore how users searching for religious content can instead encounter sexually explicit material.
• The study examines recommendation systems, community guidelines, and user experiences to understand this issue.
• The findings have implications for improving search quality and content moderation for low-resourced languages online.

Plain English Explanation

The paper looks at a problem that can happen when people search for things online in languages that don't have as much digital content available, like the Amharic language. The researchers found that people searching for religious songs in Amharic on YouTube were sometimes recommended or shown videos with sexual content instead. This can be upsetting and problematic.

The researchers investigated this issue by looking at how YouTube's recommendation system works, the guidelines YouTube has for what content is allowed, and the experiences of users searching for this kind of content. They wanted to understand why this was happening and find ways to improve the situation.

The findings from this study are important because they highlight challenges with providing safe, high-quality search results for people using languages that don't have as much digital content available. This is an issue that many papers have explored for low-resourced languages online. Addressing problems like this can help make the internet a more inclusive and helpful place for people around the world.

Technical Explanation

The paper investigates the phenomenon of users searching for religious content in the Amharic language on YouTube, but instead being recommended or shown sexually explicit material. The researchers used a mixed-methods approach, combining quantitative analysis of the YouTube recommendation system and qualitative interviews with Amharic-speaking users.

Quantitatively, the team analyzed the YouTube search and recommendation algorithms for Amharic queries related to religious topics. They found that the systems frequently surfaced inappropriate content, likely due to the lack of high-quality Amharic data that the algorithms could draw upon. This aligns with prior research on the challenges of building effective NLP systems for low-resourced languages.

Qualitatively, the researchers conducted semi-structured interviews with 15 Amharic-speaking YouTube users. Participants described their frustration and distress at encountering sexually explicit content when searching for religious material. Some noted feeling that their cultural and religious norms were being violated. These findings echo research on the detection and mitigation of gendered abuse in low-resourced languages and the challenges of defining and enforcing boundaries for offensive speech.

Critical Analysis

The paper provides valuable insights into the practical challenges of content moderation and recommendation systems for low-resourced languages like Amharic. However, it is limited in scope to a single language and platform. Further research is needed to understand whether these issues generalize to other low-resourced languages and online environments.

Additionally, the study does not delve deeply into the potential societal impacts of this problem. For example, the researchers do not explore how encountering inappropriate content when searching for religious material could affect individual users' trust in technology or their sense of online safety. Nor do they consider broader questions of digital inequity and the need for more inclusive internet experiences globally.

It would be interesting for future work to examine toxicity and harmful content across a wider range of topics and platforms, as well as to investigate solutions that bridge social and language barriers online. Nonetheless, this paper makes an important contribution by bringing attention to an underexplored area of online harm.

Conclusion

This paper sheds light on the problem of users in low-resourced language communities encountering inappropriate content when searching for benign material online. The researchers found that the combination of sparse data, imperfect recommendation algorithms, and insufficient content moderation can lead to harmful experiences for Amharic-speaking YouTube users.

The findings highlight the need for more inclusive and culturally-aware approaches to developing search and recommendation systems, as well as strengthening content moderation practices, for underserved language communities. Addressing these challenges could help make the internet a safer and more trustworthy resource for people around the world.