Automatic Textual Normalization for Hate Speech Detection

Read original: arXiv:2311.06851 - Published 7/26/2024 by Anh Thi-Hoang Nguyen, Dung Ha Nguyen, Nguyet Thi Nguyen, Khanh Thanh-Duy Ho, Kiet Van Nguyen

🗣️

Overview

Social media data is valuable for research, but contains many non-standard words.
These irregularities hinder the effectiveness of natural language processing (NLP) tools.
Current methods for the Vietnamese language involve creating manual rules or complex deep learning frameworks.
This research proposes a simpler approach using a sequence-to-sequence (Seq2Seq) model.

Plain English Explanation

The paper discusses the challenge of textual normalization in social media data, which is important for improving the performance of natural language processing (NLP) tools. Social media platforms often contain many non-standard words (NSW), such as slang, misspellings, and abbreviations, which can confuse NLP algorithms.

The researchers propose a simpler approach to this problem, using a sequence-to-sequence (Seq2Seq) model to normalize the text. This involves training the model to convert non-standard words into their correct forms. The researchers also provide a dataset of 2,181 human-annotated Vietnamese comments to support this task.

The results show that the Seq2Seq model can achieve an accuracy just under 70% for textual normalization. While this may not be the highest accuracy, the researchers demonstrate that this normalization can still improve the performance of a hate speech detection task by around 2%. This suggests that the approach has potential to enhance the effectiveness of various NLP applications.

Technical Explanation

The researchers address the issue of textual normalization in social media data, which is crucial for improving the performance of natural language processing (NLP) tasks. They note that social media platforms often contain many non-standard words (NSW), such as slang, misspellings, and abbreviations, which can confuse NLP algorithms.

In contrast to the current state-of-the-art methods for the Vietnamese language, which involve creating manual rules or implementing complex deep learning frameworks, the researchers propose a simpler approach using a sequence-to-sequence (Seq2Seq) model. This model is trained to convert non-standard words into their correct forms, effectively normalizing the text.

To support this task, the researchers provide a dataset of 2,181 human-annotated Vietnamese comments with an inter-annotator agreement of 0.9014. The results show that the Seq2Seq model can achieve an accuracy just under 70% for textual normalization.

While this accuracy may not be the highest, the researchers demonstrate that this normalization can still improve the performance of a hate speech detection task by around 2%. This suggests that the proposed approach has the potential to enhance the effectiveness of various NLP applications.

Critical Analysis

The researchers acknowledge that the accuracy of their Seq2Seq model for textual normalization, while showing promise, falls slightly short of 70%. This suggests that there may be room for improvement in the model's performance.

Additionally, the researchers do not provide a detailed comparison of their approach to the current state-of-the-art methods for the Vietnamese language, which involve creating manual rules or implementing complex deep learning frameworks. It would be helpful to understand the trade-offs between the simplicity of the Seq2Seq approach and the potential accuracy improvements of the more intricate methods.

Furthermore, the researchers only demonstrate the impact of textual normalization on a single NLP task, namely hate speech detection. It would be valuable to explore the effects of normalization on a wider range of NLP applications, such as sentiment analysis, named entity recognition, or machine translation, to better understand the broader implications of this approach.

Overall, the researchers have presented a promising and straightforward approach to addressing the challenge of textual normalization in social media data. However, further research and comparisons could help to strengthen the case for this method and its potential to improve the performance of various NLP tasks.

Conclusion

This research paper proposes a straightforward approach to the problem of textual normalization in social media data, using a sequence-to-sequence (Seq2Seq) model to convert non-standard words into their correct forms. While the accuracy of the Seq2Seq model falls slightly short of 70%, the researchers demonstrate that this normalization can still enhance the performance of a hate speech detection task by approximately 2%.

The researchers also provide a valuable dataset of 2,181 human-annotated Vietnamese comments to support this task, which can be a valuable resource for future research. Overall, this work highlights the potential of the Seq2Seq approach to address the challenges posed by non-standard words in social media data and improve the effectiveness of natural language processing (NLP) applications.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

🗣️

Automatic Textual Normalization for Hate Speech Detection

Anh Thi-Hoang Nguyen, Dung Ha Nguyen, Nguyet Thi Nguyen, Khanh Thanh-Duy Ho, Kiet Van Nguyen

Social media data is a valuable resource for research, yet it contains a wide range of non-standard words (NSW). These irregularities hinder the effective operation of NLP tools. Current state-of-the-art methods for the Vietnamese language address this issue as a problem of lexical normalization, involving the creation of manual rules or the implementation of multi-staged deep learning frameworks, which necessitate extensive efforts to craft intricate rules. In contrast, our approach is straightforward, employing solely a sequence-to-sequence (Seq2Seq) model. In this research, we provide a dataset for textual normalization, comprising 2,181 human-annotated comments with an inter-annotator agreement of 0.9014. By leveraging the Seq2Seq model for textual normalization, our results reveal that the accuracy achieved falls slightly short of 70%. Nevertheless, textual normalization enhances the accuracy of the Hate Speech Detection (HSD) task by approximately 2%, demonstrating its potential to improve the performance of complex NLP tasks. Our dataset is accessible for research purposes.

7/26/2024

Exploiting Hatred by Targets for Hate Speech Detection on Vietnamese Social Media Texts

Cuong Nhat Vo, Khanh Bao Huynh, Son T. Luu, Trong-Hop Do

The growth of social networks makes toxic content spread rapidly. Hate speech detection is a task to help decrease the number of harmful comments. With the diversity in the hate speech created by users, it is necessary to interpret the hate speech besides detecting it. Hence, we propose a methodology to construct a system for targeted hate speech detection from online streaming texts from social media. We first introduce the ViTHSD - a targeted hate speech detection dataset for Vietnamese Social Media Texts. The dataset contains 10K comments, each comment is labeled to specific targets with three levels: clean, offensive, and hate. There are 5 targets in the dataset, and each target is labeled with the corresponding level manually by humans with strict annotation guidelines. The inter-annotator agreement obtained from the dataset is 0.45 by Cohen's Kappa index, which is indicated as a moderate level. Then, we construct a baseline for this task by combining the Bi-GRU-LSTM-CNN with the pre-trained language model to leverage the power of text representation of BERTology. Finally, we suggest a methodology to integrate the baseline model for targeted hate speech detection into the online streaming system for practical application in preventing hateful and offensive content on social media.

5/1/2024

🗣️

ViHateT5: Enhancing Hate Speech Detection in Vietnamese With A Unified Text-to-Text Transformer Model

Luan Thanh Nguyen

Recent advancements in hate speech detection (HSD) in Vietnamese have made significant progress, primarily attributed to the emergence of transformer-based pre-trained language models, particularly those built on the BERT architecture. However, the necessity for specialized fine-tuned models has resulted in the complexity and fragmentation of developing a multitasking HSD system. Moreover, most current methodologies focus on fine-tuning general pre-trained models, primarily trained on formal textual datasets like Wikipedia, which may not accurately capture human behavior on online platforms. In this research, we introduce ViHateT5, a T5-based model pre-trained on our proposed large-scale domain-specific dataset named VOZ-HSD. By harnessing the power of a text-to-text architecture, ViHateT5 can tackle multiple tasks using a unified model and achieve state-of-the-art performance across all standard HSD benchmarks in Vietnamese. Our experiments also underscore the significance of label distribution in pre-training data on model efficacy. We provide our experimental materials for research purposes, including the VOZ-HSD dataset, pre-trained checkpoint, the unified HSD-multitask ViHateT5 model, and related source code on GitHub publicly.

6/5/2024

NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative Data

Manuel Tonneau, Pedro Vitor Quinta de Castro, Karim Lasri, Ibrahim Farouq, Lakshminarayanan Subramanian, Victor Orozco-Olvera, Samuel P. Fraiberger

To address the global issue of online hate, hate speech detection (HSD) systems are typically developed on datasets from the United States, thereby failing to generalize to English dialects from the Majority World. Furthermore, HSD models are often evaluated on non-representative samples, raising concerns about overestimating model performance in real-world settings. In this work, we introduce NaijaHate, the first dataset annotated for HSD which contains a representative sample of Nigerian tweets. We demonstrate that HSD evaluated on biased datasets traditionally used in the literature consistently overestimates real-world performance by at least two-fold. We then propose NaijaXLM-T, a pretrained model tailored to the Nigerian Twitter context, and establish the key role played by domain-adaptive pretraining and finetuning in maximizing HSD performance. Finally, owing to the modest performance of HSD systems in real-world conditions, we find that content moderators would need to review about ten thousand Nigerian tweets flagged as hateful daily to moderate 60% of all hateful content, highlighting the challenges of moderating hate speech at scale as social media usage continues to grow globally. Taken together, these results pave the way towards robust HSD systems and a better protection of social media users from hateful content in low-resource settings.

6/26/2024