Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities
2406.07353
0
0
Abstract
Internet memes, channels for humor, social commentary, and cultural expression, are increasingly used to spread toxic messages. Studies on the computational analyses of toxic memes have significantly grown over the past five years, and the only three surveys on computational toxic meme analysis cover only work published until 2022, leading to inconsistent terminology and unexplored trends. Our work fills this gap by surveying content-based computational perspectives on toxic memes, and reviewing key developments until early 2024. Employing the PRISMA methodology, we systematically extend the previously considered papers, achieving a threefold result. First, we survey 119 new papers, analyzing 158 computational works focused on content-based toxic meme analysis. We identify over 30 datasets used in toxic meme analysis and examine their labeling systems. Second, after observing the existence of unclear definitions of meme toxicity in computational works, we introduce a new taxonomy for categorizing meme toxicity types. We also note an expansion in computational tasks beyond the simple binary classification of memes as toxic or non-toxic, indicating a shift towards achieving a nuanced comprehension of toxicity. Third, we identify three content-based dimensions of meme toxicity under automatic study: target, intent, and conveyance tactics. We develop a framework illustrating the relationships between these dimensions and meme toxicities. The survey analyzes key challenges and recent trends, such as enhanced cross-modal reasoning, integrating expert and cultural knowledge, the demand for automatic toxicity explanations, and handling meme toxicity in low-resource languages. Also, it notes the rising use of Large Language Models (LLMs) and generative AI for detecting and generating toxic memes. Finally, it proposes pathways for advancing toxic meme detection and interpretation.
Create account to get full access
Overview
- This paper provides a comprehensive survey of computational approaches for detecting and explaining the toxicity of internet memes.
- The authors examine various techniques used to identify harmful or undesirable content in memes, as well as methods for understanding the underlying causes and social impacts of toxic memes.
- The paper covers a range of topics, including analyzing toxicity in deep conversations on Reddit, detecting propagandistic content in Arabic memes, and hierarchical classification of toxic memes.
Plain English Explanation
Internet memes are a popular form of online content that can sometimes contain harmful or toxic elements. This paper explores computational methods for identifying and understanding these toxic memes. The researchers examine various techniques used to detect toxic content, such as analyzing the language and imagery in memes, as well as approaches for explaining the underlying causes and social impacts of these problematic memes.
The paper covers a range of related research, including studies on the toxicity of content across different topics on Reddit and efforts to go beyond traditional toxicity detection by considering more nuanced factors. By surveying this diverse body of work, the authors provide a comprehensive overview of the computational perspectives on the challenge of toxic memes.
Technical Explanation
The paper begins by introducing the concept of toxic memes and the importance of understanding and mitigating their impact on online communities. The authors then provide a detailed review of the existing research in this area, covering a wide range of computational approaches.
One key focus of the paper is the detection of toxic memes. The researchers examine various machine learning and natural language processing techniques that have been used to identify harmful or undesirable content in memes, including analyzing the linguistic and visual features of memes and leveraging hierarchical classification models.
The paper also explores methods for explaining the underlying causes and social impacts of toxic memes. This includes studies on the propagation of propagandistic content in Arabic memes and efforts to analyze the toxicity of content across different topics and communities on platforms like Reddit.
Throughout the survey, the authors highlight the key insights, challenges, and future research directions in this rapidly evolving field.
Critical Analysis
The paper provides a comprehensive and well-researched overview of the computational approaches to detecting and explaining the toxicity of internet memes. The authors have done an excellent job of covering a diverse range of relevant studies and techniques, offering readers a thorough understanding of the current state of the art in this field.
One potential area for further exploration is the consideration of more nuanced and context-dependent factors in the detection and explanation of toxic memes. As mentioned in the paper, some researchers have begun to move beyond traditional toxicity detection methods by incorporating additional factors, such as the intent and social impact of the content. Expanding this line of research could lead to more robust and meaningful insights into the complex dynamics of toxic memes.
Additionally, the paper could have delved deeper into the potential ethical and privacy concerns associated with the computational analysis of memes, particularly in terms of data collection, algorithmic biases, and the potential misuse of these technologies. Addressing these issues would strengthen the paper's critical analysis and provide a more well-rounded perspective on the challenges and limitations of this research.
Conclusion
This paper provides a comprehensive survey of the computational perspectives on the detection and explanation of toxic memes. The authors have done an excellent job of synthesizing a diverse range of research, covering key techniques, insights, and future directions in this rapidly evolving field.
The findings presented in this paper have important implications for online content moderation, digital literacy, and the broader understanding of the social impact of internet memes. By continuing to advance the computational approaches to tackling toxic memes, researchers and practitioners can work towards creating safer and more inclusive online communities.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
OSPC: Detecting Harmful Memes with Large Language Model as a Catalyst
Jingtao Cao, Zheng Zhang, Hongru Wang, Bin Liang, Hao Wang, Kam-Fai Wong
0
0
Memes, which rapidly disseminate personal opinions and positions across the internet, also pose significant challenges in propagating social bias and prejudice. This study presents a novel approach to detecting harmful memes, particularly within the multicultural and multilingual context of Singapore. Our methodology integrates image captioning, Optical Character Recognition (OCR), and Large Language Model (LLM) analysis to comprehensively understand and classify harmful memes. Utilizing the BLIP model for image captioning, PP-OCR and TrOCR for text recognition across multiple languages, and the Qwen LLM for nuanced language understanding, our system is capable of identifying harmful content in memes created in English, Chinese, Malay, and Tamil. To enhance the system's performance, we fine-tuned our approach by leveraging additional data labeled using GPT-4V, aiming to distill the understanding capability of GPT-4V for harmful memes to our system. Our framework achieves top-1 at the public leaderboard of the Online Safety Prize Challenge hosted by AI Singapore, with the AUROC as 0.7749 and accuracy as 0.7087, significantly ahead of the other teams. Notably, our approach outperforms previous benchmarks, with FLAVA achieving an AUROC of 0.5695 and VisualBERT an AUROC of 0.5561.
6/17/2024
Analyzing Toxicity in Deep Conversations: A Reddit Case Study
Vigneshwaran Shankaran, Rajesh Sharma
0
0
Online social media has become increasingly popular in recent years due to its ease of access and ability to connect with others. One of social media's main draws is its anonymity, allowing users to share their thoughts and opinions without fear of judgment or retribution. This anonymity has also made social media prone to harmful content, which requires moderation to ensure responsible and productive use. Several methods using artificial intelligence have been employed to detect harmful content. However, conversation and contextual analysis of hate speech are still understudied. Most promising works only analyze a single text at a time rather than the conversation supporting it. In this work, we employ a tree-based approach to understand how users behave concerning toxicity in public conversation settings. To this end, we collect both the posts and the comment sections of the top 100 posts from 8 Reddit communities that allow profanity, totaling over 1 million responses. We find that toxic comments increase the likelihood of subsequent toxic comments being produced in online conversations. Our analysis also shows that immediate context plays a vital role in shaping a response rather than the original post. We also study the effect of consensual profanity and observe overlapping similarities with non-consensual profanity in terms of user behavior and patterns.
4/12/2024
ArMeme: Propagandistic Content in Arabic Memes
Firoj Alam, Abul Hasnat, Fatema Ahmed, Md Arid Hasan, Maram Hasanain
0
0
With the rise of digital communication, memes have become a significant medium for cultural and political expression that is often used to mislead audiences. Identification of such misleading and persuasive multimodal content has become more important among various stakeholders, including social media platforms, policymakers, and the broader society as they often cause harm to individuals, organizations, and/or society. While there has been effort to develop AI-based automatic systems for resource-rich languages (e.g., English), it is relatively little to none for medium to low resource languages. In this study, we focused on developing an Arabic memes dataset with manual annotations of propagandistic content. We annotated ~6K Arabic memes collected from various social media platforms, which is a first resource for Arabic multimodal research. We provide a comprehensive analysis aiming to develop computational tools for their detection. We will make them publicly available for the community.
6/7/2024
IITK at SemEval-2024 Task 4: Hierarchical Embeddings for Detection of Persuasion Techniques in Memes
Shreenaga Chikoti, Shrey Mehta, Ashutosh Modi
0
0
Memes are one of the most popular types of content used in an online disinformation campaign. They are primarily effective on social media platforms since they can easily reach many users. Memes in a disinformation campaign achieve their goal of influencing the users through several rhetorical and psychological techniques, such as causal oversimplification, name-calling, and smear. The SemEval 2024 Task 4 textit{Multilingual Detection of Persuasion Technique in Memes} on identifying such techniques in the memes is divided across three sub-tasks: ($mathbf{1}$) Hierarchical multi-label classification using only textual content of the meme, ($mathbf{2}$) Hierarchical multi-label classification using both, textual and visual content of the meme and ($mathbf{3}$) Binary classification of whether the meme contains a persuasion technique or not using it's textual and visual content. This paper proposes an ensemble of Class Definition Prediction (CDP) and hyperbolic embeddings-based approaches for this task. We enhance meme classification accuracy and comprehensiveness by integrating HypEmo's hierarchical label embeddings (Chen et al., 2023) and a multi-task learning framework for emotion prediction. We achieve a hierarchical F1-score of 0.60, 0.67, and 0.48 on the respective sub-tasks.
4/9/2024