OPSD: an Offensive Persian Social media Dataset and its baseline evaluations
0
Sign in to get full access
Overview
- This paper introduces a new dataset called the Offensive Persian Social media Dataset (OPSD) for studying offensive language in Persian-language social media.
- The paper also presents baseline evaluations of several machine learning models for detecting offensive content in the OPSD dataset.
Plain English Explanation
This research paper focuses on creating and analyzing a new dataset for studying offensive language on Persian-language social media platforms. The researchers developed the Offensive Persian Social media Dataset (OPSD), which contains a large collection of posts and comments from social media that have been labeled as either offensive or not offensive.
The researchers then tested how well different machine learning models could identify offensive content in the OPSD dataset. This type of technology could be useful for automatically detecting and moderating offensive or abusive language online, especially on social media platforms that host content in non-English languages like Persian.
The creation of the OPSD dataset and the baseline evaluations of machine learning models provide a valuable resource for researchers and developers working on addressing online hate speech and offensive content, particularly in the context of Persian-language social media.
Technical Explanation
The paper introduces the Offensive Persian Social media Dataset (OPSD), a new dataset for studying offensive language on Persian-language social media platforms. The dataset contains over 50,000 Persian-language social media posts and comments that have been manually annotated as either offensive or non-offensive.
To establish baseline performance, the researchers evaluated several machine learning models on the OPSD dataset, including models described in this paper, this paper, and this paper. The models were tested on their ability to accurately classify the social media content as offensive or not.
The results of the baseline evaluations provide a starting point for future research using the OPSD dataset, which the authors hope will enable more work on detecting and mitigating online offensive content in Persian-language social media.
Critical Analysis
The creation of the OPSD dataset is a valuable contribution to the field of online hate speech detection, as it provides a resource for studying offensive language in a non-English language context. However, the paper does not provide much detail on the annotation process or the demographic characteristics of the data, which could impact the generalizability of the findings.
Additionally, the baseline evaluations presented in the paper are fairly limited, focusing only on a few existing machine learning models. It would be interesting to see how more advanced techniques, such as the methods described in this paper or the approach discussed in this paper, perform on the OPSD dataset.
Overall, this research represents an important step in addressing the challenge of online offensive content, particularly in non-English language contexts. However, further work is needed to fully understand the nuances and complexities of this issue.
Conclusion
This paper introduces a new dataset, the Offensive Persian Social media Dataset (OPSD), and presents baseline evaluations of several machine learning models for detecting offensive content in Persian-language social media posts. The creation of the OPSD dataset and the initial model evaluations provide a valuable resource for researchers and developers working on addressing online hate speech and offensive content, particularly in the context of Persian-language social media. While the research represents an important step forward, further work is needed to fully understand and address the complex challenge of online offensive content.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
OPSD: an Offensive Persian Social media Dataset and its baseline evaluations
Mehran Safayani, Amir Sartipi, Amir Hossein Ahmadi, Parniyan Jalali, Amir Hossein Mansouri, Mohammad Bisheh-Niasar, Zahra Pourbahman
The proliferation of hate speech and offensive comments on social media has become increasingly prevalent due to user activities. Such comments can have detrimental effects on individuals' psychological well-being and social behavior. While numerous datasets in the English language exist in this domain, few equivalent resources are available for Persian language. To address this gap, this paper introduces two offensive datasets. The first dataset comprises annotations provided by domain experts, while the second consists of a large collection of unlabeled data obtained through web crawling for unsupervised learning purposes. To ensure the quality of the former dataset, a meticulous three-stage labeling process was conducted, and kappa measures were computed to assess inter-annotator agreement. Furthermore, experiments were performed on the dataset using state-of-the-art language models, both with and without employing masked language modeling techniques, as well as machine learning algorithms, in order to establish the baselines for the dataset using contemporary cutting-edge approaches. The obtained F1-scores for the three-class and two-class versions of the dataset were 76.9% and 89.9% for XLM-RoBERTa, respectively.
Read more4/9/2024
0
Towards Generalized Offensive Language Identification
Alphaeus Dmonte, Tejas Arya, Tharindu Ranasinghe, Marcos Zampieri
The prevalence of offensive content on the internet, encompassing hate speech and cyberbullying, is a pervasive issue worldwide. Consequently, it has garnered significant attention from the machine learning (ML) and natural language processing (NLP) communities. As a result, numerous systems have been developed to automatically identify potentially harmful content and mitigate its impact. These systems can follow two approaches; (1) Use publicly available models and application endpoints, including prompting large language models (LLMs) (2) Annotate datasets and train ML models on them. However, both approaches lack an understanding of how generalizable they are. Furthermore, the applicability of these systems is often questioned in off-domain and practical environments. This paper empirically evaluates the generalizability of offensive language detection models and datasets across a novel generalized benchmark. We answer three research questions on generalizability. Our findings will be useful in creating robust real-world offensive language detection systems.
Read more7/29/2024
💬
0
A multilingual dataset for offensive language and hate speech detection for hausa, yoruba and igbo languages
Saminu Mohammad Aliyu, Gregory Maksha Wajiga, Muhammad Murtala
The proliferation of online offensive language necessitates the development of effective detection mechanisms, especially in multilingual contexts. This study addresses the challenge by developing and introducing novel datasets for offensive language detection in three major Nigerian languages: Hausa, Yoruba, and Igbo. We collected data from Twitter and manually annotated it to create datasets for each of the three languages, using native speakers. We used pre-trained language models to evaluate their efficacy in detecting offensive language in our datasets. The best-performing model achieved an accuracy of 90%. To further support research in offensive language detection, we plan to make the dataset and our models publicly available.
Read more6/7/2024
0
IndoToxic2024: A Demographically-Enriched Dataset of Hate Speech and Toxicity Types for Indonesian Language
Lucky Susanto, Musa Izzanardi Wijanarko, Prasetia Anugrah Pratama, Traci Hong, Ika Idris, Alham Fikri Aji, Derry Wijaya
Hate speech poses a significant threat to social harmony. Over the past two years, Indonesia has seen a ten-fold increase in the online hate speech ratio, underscoring the urgent need for effective detection mechanisms. However, progress is hindered by the limited availability of labeled data for Indonesian texts. The condition is even worse for marginalized minorities, such as Shia, LGBTQ, and other ethnic minorities because hate speech is underreported and less understood by detection tools. Furthermore, the lack of accommodation for subjectivity in current datasets compounds this issue. To address this, we introduce IndoToxic2024, a comprehensive Indonesian hate speech and toxicity classification dataset. Comprising 43,692 entries annotated by 19 diverse individuals, the dataset focuses on texts targeting vulnerable groups in Indonesia, specifically during the hottest political event in the country: the presidential election. We establish baselines for seven binary classification tasks, achieving a macro-F1 score of 0.78 with a BERT model (IndoBERTweet) fine-tuned for hate speech classification. Furthermore, we demonstrate how incorporating demographic information can enhance the zero-shot performance of the large language model, gpt-3.5-turbo. However, we also caution that an overemphasis on demographic information can negatively impact the fine-tuned model performance due to data fragmentation.
Read more6/28/2024