ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media
2305.14225
0
0
🔄
Abstract
Considerable advancements have been made to tackle the misrepresentation of information derived from reference articles in the domains of fact-checking and faithful summarization. However, an unaddressed aspect remains - the identification of social media posts that manipulate information within associated news articles. This task presents a significant challenge, primarily due to the prevalence of personal opinions in such posts. We present a novel task, identifying manipulation of news on social media, which aims to detect manipulation in social media posts and identify manipulated or inserted information. To study this task, we have proposed a data collection schema and curated a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles. Our analysis demonstrates that this task is highly challenging, with large language models (LLMs) yielding unsatisfactory performance. Additionally, we have developed a simple yet effective basic model that outperforms LLMs significantly on the ManiTweet dataset. Finally, we have conducted an exploratory analysis of human-written tweets, unveiling intriguing connections between manipulation and the domain and factuality of news articles, as well as revealing that manipulated sentences are more likely to encapsulate the main story or consequences of a news outlet.
Create account to get full access
Overview
- Researchers have made progress in fact-checking and faithful summarization, but an important aspect remains unaddressed: identifying social media posts that manipulate information from associated news articles.
- This task is challenging due to the prevalence of personal opinions in such posts.
- The researchers propose a novel task, "Identifying Manipulation of News on Social Media," which aims to detect manipulation in social media posts and identify manipulated or inserted information.
- To study this task, the researchers have developed a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles.
Plain English Explanation
Fact-checking and summarizing information from reference articles have seen significant advancements. However, one aspect that remains unaddressed is the identification of social media posts that manipulate the information within associated news articles. This is a challenging task because social media posts often contain personal opinions, making it difficult to distinguish legitimate information from manipulated content.
To address this problem, the researchers have proposed a new task called "Identifying Manipulation of News on Social Media." The goal is to detect when social media posts, specifically tweets, are manipulating or inserting information from the news articles they reference. To study this task, the researchers have created a dataset called ManiTweet, which contains 3.6K pairs of tweets and their corresponding news articles.
Technical Explanation
The researchers have developed a novel task called "Identifying Manipulation of News on Social Media," which aims to detect manipulation in social media posts and identify manipulated or inserted information. To study this task, they have proposed a data collection schema and curated the ManiTweet dataset, which consists of 3.6K pairs of tweets and their corresponding news articles.
The researchers' analysis of this dataset demonstrates that this task is highly challenging, with large language models (LLMs) yielding unsatisfactory performance. To address this, the researchers have developed a simple yet effective basic model that outperforms LLMs significantly on the ManiTweet dataset.
Additionally, the researchers have conducted an exploratory analysis of human-written tweets, revealing intriguing connections between manipulation and the domain and factuality of news articles. They have also found that manipulated sentences are more likely to encapsulate the main story or consequences of a news outlet.
Critical Analysis
The researchers have identified an important and underexplored aspect of information manipulation - the identification of social media posts that manipulate information within associated news articles. This task presents unique challenges due to the prevalence of personal opinions in such posts, which can make it difficult to distinguish legitimate information from manipulated content.
While the researchers have developed a novel dataset (ManiTweet) and a basic model that outperforms large language models, the overall performance on this task remains unsatisfactory. This highlights the need for further research and advancements in adapting fake news detection to the era of large language models and exploring the potential of large language models (LLMs) for identifying manipulation.
Additionally, the researchers' exploratory analysis of human-written tweets provides valuable insights into the connections between manipulation and the domain and factuality of news articles, as well as the likelihood of manipulated sentences encapsulating the main story or consequences of a news outlet. These findings could inform future approaches to analyzing disinformation narratives and exposing and explaining fake news.
Conclusion
This research presents a novel task of identifying manipulation of news on social media, which addresses an important and underexplored aspect of information manipulation. The researchers have developed a dataset (ManiTweet) and a basic model to tackle this challenge, highlighting the significant difficulties in this domain. The insights gained from the exploratory analysis of human-written tweets provide valuable directions for future research in detecting and mitigating the spread of manipulated information on social media platforms.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Deepfake tweets automatic detection
Adam Frej, Adrian Kaminski, Piotr Marciniak, Szymon Szmajdzinski, Soveatin Kuntur, Anna Wroblewska
0
0
This study addresses the critical challenge of detecting DeepFake tweets by leveraging advanced natural language processing (NLP) techniques to distinguish between genuine and AI-generated texts. Given the increasing prevalence of misinformation, our research utilizes the TweepFake dataset to train and evaluate various machine learning models. The objective is to identify effective strategies for recognizing DeepFake content, thereby enhancing the integrity of digital communications. By developing reliable methods for detecting AI-generated misinformation, this work contributes to a more trustworthy online information environment.
6/26/2024
MentalManip: A Dataset For Fine-grained Analysis of Mental Manipulation in Conversations
Yuxin Wang, Ivory Yang, Saeed Hassanpour, Soroush Vosoughi
0
0
Mental manipulation, a significant form of abuse in interpersonal conversations, presents a challenge to identify due to its context-dependent and often subtle nature. The detection of manipulative language is essential for protecting potential victims, yet the field of Natural Language Processing (NLP) currently faces a scarcity of resources and research on this topic. Our study addresses this gap by introducing a new dataset, named ${rm M{small ental}M{small anip}}$, which consists of $4,000$ annotated movie dialogues. This dataset enables a comprehensive analysis of mental manipulation, pinpointing both the techniques utilized for manipulation and the vulnerabilities targeted in victims. Our research further explores the effectiveness of leading-edge models in recognizing manipulative dialogue and its components through a series of experiments with various configurations. The results demonstrate that these models inadequately identify and categorize manipulative content. Attempts to improve their performance by fine-tuning with existing datasets on mental health and toxicity have not overcome these limitations. We anticipate that ${rm M{small ental}M{small anip}}$ will stimulate further research, leading to progress in both understanding and mitigating the impact of mental manipulation in conversations.
5/28/2024
Seeing Through AI's Lens: Enhancing Human Skepticism Towards LLM-Generated Fake News
Navid Ayoobi, Sadat Shahriar, Arjun Mukherjee
0
0
LLMs offer valuable capabilities, yet they can be utilized by malicious users to disseminate deceptive information and generate fake news. The growing prevalence of LLMs poses difficulties in crafting detection approaches that remain effective across various text domains. Additionally, the absence of precautionary measures for AI-generated news on online social platforms is concerning. Therefore, there is an urgent need to improve people's ability to differentiate between news articles written by humans and those produced by LLMs. By providing cues in human-written and LLM-generated news, we can help individuals increase their skepticism towards fake LLM-generated news. This paper aims to elucidate simple markers that help individuals distinguish between articles penned by humans and those created by LLMs. To achieve this, we initially collected a dataset comprising 39k news articles authored by humans or generated by four distinct LLMs with varying degrees of fake. We then devise a metric named Entropy-Shift Authorship Signature (ESAS) based on the information theory and entropy principles. The proposed ESAS ranks terms or entities, like POS tagging, within news articles based on their relevance in discerning article authorship. We demonstrate the effectiveness of our metric by showing the high accuracy attained by a basic method, i.e., TF-IDF combined with logistic regression classifier, using a small set of terms with the highest ESAS score. Consequently, we introduce and scrutinize these top ESAS-ranked terms to aid individuals in strengthening their skepticism towards LLM-generated fake news.
6/21/2024
🔎
Adapting Fake News Detection to the Era of Large Language Models
Jinyan Su, Claire Cardie, Preslav Nakov
0
0
In the age of large language models (LLMs) and the widespread adoption of AI-driven content creation, the landscape of information dissemination has witnessed a paradigm shift. With the proliferation of both human-written and machine-generated real and fake news, robustly and effectively discerning the veracity of news articles has become an intricate challenge. While substantial research has been dedicated to fake news detection, this either assumes that all news articles are human-written or abruptly assumes that all machine-generated news are fake. Thus, a significant gap exists in understanding the interplay between machine-(paraphrased) real news, machine-generated fake news, human-written fake news, and human-written real news. In this paper, we study this gap by conducting a comprehensive evaluation of fake news detectors trained in various scenarios. Our primary objectives revolve around the following pivotal question: How to adapt fake news detectors to the era of LLMs? Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa. Moreover, due to the bias of detectors against machine-generated texts cite{su2023fake}, they should be trained on datasets with a lower machine-generated news ratio than the test set. Building on our findings, we provide a practical strategy for the development of robust fake news detectors.
4/16/2024