Assessing Large Language Models for Online Extremism Research: Identification, Explanation, and New Knowledge
0
💬
Sign in to get full access
Overview
- The United States has seen a significant rise in violent extremism, leading to the need for automated tools to detect and limit the spread of extremist ideology online.
- This study evaluates the performance of Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformers (GPT) in detecting and classifying online domestic extremist posts.
- Researchers collected social media posts containing far-right and far-left ideological keywords and manually labeled them as extremist or non-extremist.
- Extremist posts were further classified into one or more of five contributing elements of extremism.
- The study compares the performance of GPT 3.5 and GPT 4 models using different prompts, and also examines knowledge transfer between BERT model training data sizes and categories.
Plain English Explanation
The paper looks at using artificial intelligence (AI) models to automatically identify and classify posts on social media that contain extremist content. Extremism, especially in the far-right and far-left political spectrum, has been on the rise in the United States, and there is a need for tools to detect and limit the spread of these harmful ideologies online.
The researchers used two popular AI models, BERT and GPT, to analyze social media posts and classify them as either extremist or non-extremist. They also tried to further categorize the extremist posts based on different elements that contribute to extremism. The study compared how well the BERT and GPT models performed at this task, and looked at factors like the amount of training data and the specific instructions (prompts) given to the GPT models.
The key finding was that the GPT models, especially the more advanced GPT-4, tended to outperform the BERT models in accurately identifying and classifying extremist content. However, the researchers also found that the GPT models had some unique sensitivities, with GPT-3.5 doing better at detecting far-left extremism and GPT-4 performing better on far-right extremism.
Overall, the research suggests that large language models like GPT hold a lot of potential for automating the detection and classification of online extremism, but more work is needed to optimize and refine these tools to make them as accurate and efficient as possible.
Technical Explanation
The researchers collected a dataset of social media posts containing keywords associated with far-right and far-left ideologies. They manually labeled these posts as either extremist or non-extremist, and further classified the extremist posts into one or more of five contributing elements of extremism.
They then evaluated the performance of two prominent AI language models, BERT and GPT, in detecting and classifying this extremist content. For the BERT models, they examined how the model's performance was affected by the size of the training dataset and by knowledge transfer between different categories of extremism.
For the GPT models, the researchers compared the performance of GPT-3.5 and GPT-4 using different types of prompts: a naive prompt, a layperson-definition prompt, a role-playing prompt, and a professional-definition prompt. They found that more detailed prompts generally led to better classification results, but overly complex prompts could impair performance.
The key finding was that the best-performing GPT models outperformed the best-performing BERT models in a zero-shot setting (without any fine-tuning on the extremist dataset). The researchers also observed that the different versions of GPT had unique sensitivities, with GPT-3.5 performing better on far-left extremist posts and GPT-4 performing better on far-right extremist posts.
Overall, the study suggests that large language models like GPT hold significant potential for online extremism classification tasks, surpassing traditional BERT models. However, the researchers note that further research is needed to optimize human-computer interactions and develop more efficient and effective methods for identifying extremist content.
Critical Analysis
The researchers acknowledge several limitations and areas for further research in their study. First, they note that their dataset of extremist and non-extremist posts, while carefully curated, may not be fully representative of the broader landscape of online extremism. Expanding the dataset to include a wider range of content and platforms could help validate the findings.
Additionally, while the study compares the performance of BERT and GPT models, it does not delve into the underlying reasons for the differences in their capabilities. Exploring the specific architectural and training differences between these models, and how they contribute to extremism detection, could provide valuable insights.
The researchers also highlight the need to further investigate the unique sensitivities of different GPT versions, such as GPT-3.5's stronger performance on far-left extremism and GPT-4's advantage on far-right extremism. Understanding the factors driving these differences could help in developing more robust and versatile extremism detection systems.
Finally, the paper emphasizes the importance of optimizing human-computer interactions in the context of extremism classification tasks. Exploring how end-users, such as content moderators or law enforcement, can best leverage and provide feedback to refine these AI-powered tools could lead to more efficient and effective methods for identifying and addressing online extremism.
Conclusion
This study demonstrates the significant potential of large language models, represented by GPT, in the task of automatically detecting and classifying online extremist content. The researchers found that GPT models, particularly the more advanced GPT-4, outperformed traditional BERT models in a zero-shot setting, highlighting the rapid advancements in natural language processing capabilities.
However, the study also underscores the need for further research to optimize these tools and address their potential limitations. Expanding the dataset, exploring model architectures and training approaches, and enhancing human-computer interactions are all areas that warrant deeper investigation to develop more robust and effective systems for combating the rise of violent extremism online.
As AI technology continues to evolve, the findings of this paper suggest that large language models like GPT could play a crucial role in the ongoing effort to identify and mitigate the spread of harmful extremist ideologies, ultimately contributing to a safer and more inclusive online environment.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
💬
0
Assessing Large Language Models for Online Extremism Research: Identification, Explanation, and New Knowledge
Beidi Dong, Jin R. Lee, Ziwei Zhu, Balassubramanian Srinivasan
The United States has experienced a significant increase in violent extremism, prompting the need for automated tools to detect and limit the spread of extremist ideology online. This study evaluates the performance of Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformers (GPT) in detecting and classifying online domestic extremist posts. We collected social media posts containing far-right and far-left ideological keywords and manually labeled them as extremist or non-extremist. Extremist posts were further classified into one or more of five contributing elements of extremism based on a working definitional framework. The BERT model's performance was evaluated based on training data size and knowledge transfer between categories. We also compared the performance of GPT 3.5 and GPT 4 models using different prompts: naive, layperson-definition, role-playing, and professional-definition. Results showed that the best performing GPT models outperformed the best performing BERT models, with more detailed prompts generally yielding better results. However, overly complex prompts may impair performance. Different versions of GPT have unique sensitives to what they consider extremist. GPT 3.5 performed better at classifying far-left extremist posts, while GPT 4 performed better at classifying far-right extremist posts. Large language models, represented by GPT models, hold significant potential for online extremism classification tasks, surpassing traditional BERT models in a zero-shot setting. Future research should explore human-computer interactions in optimizing GPT models for extremist detection and classification tasks to develop more efficient (e.g., quicker, less effort) and effective (e.g., fewer errors or mistakes) methods for identifying extremist content.
Read more8/30/2024
🔄
0
BERT vs GPT for financial engineering
Edward Sharkey, Philip Treleaven
The paper benchmarks several Transformer models [4], to show how these models can judge sentiment from a news event. This signal can then be used for downstream modelling and signal identification for commodity trading. We find that fine-tuned BERT models outperform fine-tuned or vanilla GPT models on this task. Transformer models have revolutionized the field of natural language processing (NLP) in recent years, achieving state-of-the-art results on various tasks such as machine translation, text summarization, question answering, and natural language generation. Among the most prominent transformer models are Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT), which differ in their architectures and objectives. A CopBERT model training data and process overview is provided. The CopBERT model outperforms similar domain specific BERT trained models such as FinBERT. The below confusion matrices show the performance on CopBERT & CopGPT respectively. We see a ~10 percent increase in f1_score when compare CopBERT vs GPT4 and 16 percent increase vs CopGPT. Whilst GPT4 is dominant It highlights the importance of considering alternatives to GPT models for financial engineering tasks, given risks of hallucinations, and challenges with interpretability. We unsurprisingly see the larger LLMs outperform the BERT models, with predictive power. In summary BERT is partially the new XGboost, what it lacks in predictive power it provides with higher levels of interpretability. Concluding that BERT models might not be the next XGboost [2], but represent an interesting alternative for financial engineering tasks, that require a blend of interpretability and accuracy.
Read more5/24/2024
0
Detecting Anti-Semitic Hate Speech using Transformer-based Large Language Models
Dengyi Liu, Minghao Wang, Andrew G. Catlin
Academic researchers and social media entities grappling with the identification of hate speech face significant challenges, primarily due to the vast scale of data and the dynamic nature of hate speech. Given the ethical and practical limitations of large predictive models like ChatGPT in directly addressing such sensitive issues, our research has explored alternative advanced transformer-based and generative AI technologies since 2019. Specifically, we developed a new data labeling technique and established a proof of concept targeting anti-Semitic hate speech, utilizing a variety of transformer models such as BERT (arXiv:1810.04805), DistillBERT (arXiv:1910.01108), RoBERTa (arXiv:1907.11692), and LLaMA-2 (arXiv:2307.09288), complemented by the LoRA fine-tuning approach (arXiv:2106.09685). This paper delineates and evaluates the comparative efficacy of these cutting-edge methods in tackling the intricacies of hate speech detection, highlighting the need for responsible and carefully managed AI applications within sensitive contexts.
Read more5/8/2024
0
LLMs left, right, and center: Assessing GPT's capabilities to label political bias from web domains
Raphael Hernandes
This research investigates whether OpenAI's GPT-4, a state-of-the-art large language model, can accurately classify the political bias of news sources based solely on their URLs. Given the subjective nature of political labels, third-party bias ratings like those from Ad Fontes Media, AllSides, and Media Bias/Fact Check (MBFC) are often used in research to analyze news source diversity. This study aims to determine if GPT-4 can replicate these human ratings on a seven-degree scale (far-left to far-right). The analysis compares GPT-4's classifications against MBFC's, and controls for website popularity using Open PageRank scores. Findings reveal a high correlation ($text{Spearman's } rho = .89$, $n = 5,877$, $p < 0.001$) between GPT-4's and MBFC's ratings, indicating the model's potential reliability. However, GPT-4 abstained from classifying approximately $frac{2}{3}$ of the dataset, particularly less popular and less biased sources. The study also identifies a slight leftward skew in GPT-4's classifications compared to MBFC's. The analysis suggests that while GPT-4 can be a scalable, cost-effective tool for political bias classification of news websites, but its use should complement human judgment to mitigate biases. Further research is recommended to explore the model's performance across different settings, languages, and additional datasets.
Read more7/22/2024