The FIGNEWS Shared Task on News Media Narratives

Read original: arXiv:2407.18147 - Published 7/26/2024 by Wajdi Zaghouani (Northwestern University in Qatar), Mustafa Jarrar (Birzeit University), Nizar Habash (New York University Abu Dhabi), Houda Bouamor (Carnegie Mellon University Qatar), Imed Zitouni (Google), Mona Diab (Carnegie Mellon University), Samhaa R. El-Beltagy (Newgiza University), Muhammed AbuOdeh (New York University Abu Dhabi)
Total Score

0

The FIGNEWS Shared Task on News Media Narratives

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • The FIGNEWS Shared Task focuses on understanding news media narratives, which is an important task for analyzing information and bias in news coverage.
  • The task involves several components, including data collection, selection, and annotation to create a high-quality dataset.
  • Participants are tasked with developing models to perform various natural language processing (NLP) tasks on this dataset, such as subjectivity detection, propaganda technique identification, and named entity recognition.

Plain English Explanation

The FIGNEWS Shared Task is an effort to better understand the way news stories are presented and framed by different media outlets. This is an important issue, as news coverage can significantly influence how people perceive events and issues.

The first step in this process is to create a high-quality dataset of news articles. Researchers collect and curate a diverse set of articles, annotating them for various linguistic and narrative features. This annotated dataset then serves as the basis for the shared task, where participants develop machine learning models to tackle challenges like identifying subjective language, detecting propaganda techniques, and recognizing named entities.

The goal is to advance the state-of-the-art in these NLP tasks, while also gaining deeper insights into how news narratives are constructed and potentially biased. By analyzing the strengths and weaknesses of different models, researchers can better understand the complexities of language use in the media and develop more robust techniques for media analysis.

Technical Explanation

The FIGNEWS Shared Task focuses on creating and analyzing a dataset of news articles to better understand media narratives. The first step involves data collection and selection, where researchers gather a diverse set of news articles from various sources and languages. This dataset is then manually annotated for a range of linguistic and narrative features, such as subjectivity, propaganda techniques, and named entities.

The annotated dataset serves as the basis for the shared task, where participants are challenged to develop machine learning models to tackle different NLP tasks. These tasks include subjectivity detection, propaganda technique identification, and named entity recognition.

By evaluating the performance of these models on the FIGNEWS dataset, researchers can gain insights into how news narratives are constructed and potentially biased. The models' strengths and weaknesses can reveal patterns in language use, sentiment, and the framing of information, which can inform further research and development of more robust media analysis techniques.

Critical Analysis

The FIGNEWS Shared Task represents an important step in advancing the understanding of news media narratives, but it also has some potential limitations and areas for further exploration.

One key challenge is the inherent subjectivity and complexity of interpreting news coverage, which can make it difficult to develop reliable and comprehensive annotation guidelines. The researchers acknowledge this issue and emphasize the need for careful curation and validation of the dataset.

Additionally, the shared task focuses primarily on textual analysis, while visual and multimedia elements in news coverage can also play a significant role in shaping narratives. Expanding the scope to include multimodal analysis could provide a more holistic understanding of news media bias and framing.

Furthermore, the current dataset is limited to a specific set of languages and sources, which may limit the generalizability of the findings. Expanding the dataset to include a broader range of languages, regions, and media types could enhance the diversity and robustness of the research.

Despite these potential limitations, the FIGNEWS Shared Task represents a valuable contribution to the field of media analysis. By fostering collaboration and competition among researchers, the task can drive innovation and push the boundaries of natural language processing and media studies.

Conclusion

The FIGNEWS Shared Task is an important effort to advance the understanding of news media narratives and bias. By creating a high-quality, annotated dataset of news articles and challenging researchers to develop advanced NLP models, the task aims to uncover the linguistic and narrative patterns that shape how information is presented and perceived.

The insights gained from this research can have far-reaching implications, informing media literacy efforts, journalistic practices, and the development of more robust tools for media analysis and fact-checking. As the field of computational media studies continues to evolve, initiatives like the FIGNEWS Shared Task will play a crucial role in advancing our understanding of the complex interplay between language, power, and the shaping of public discourse.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

The FIGNEWS Shared Task on News Media Narratives
Total Score

0

The FIGNEWS Shared Task on News Media Narratives

Wajdi Zaghouani (Northwestern University in Qatar), Mustafa Jarrar (Birzeit University), Nizar Habash (New York University Abu Dhabi), Houda Bouamor (Carnegie Mellon University Qatar), Imed Zitouni (Google), Mona Diab (Carnegie Mellon University), Samhaa R. El-Beltagy (Newgiza University), Muhammed AbuOdeh (New York University Abu Dhabi)

We present an overview of the FIGNEWS shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. The shared task addresses bias and propaganda annotation in multilingual news posts. We focus on the early days of the Israel War on Gaza as a case study. The task aims to foster collaboration in developing annotation guidelines for subjective tasks by creating frameworks for analyzing diverse narratives highlighting potential bias and propaganda. In a spirit of fostering and encouraging diversity, we address the problem from a multilingual perspective, namely within five languages: English, French, Arabic, Hebrew, and Hindi. A total of 17 teams participated in two annotation subtasks: bias (16 teams) and propaganda (6 teams). The teams competed in four evaluation tracks: guidelines development, annotation quality, annotation quantity, and consistency. Collectively, the teams produced 129,800 data points. Key findings and implications for the field are discussed.

Read more

7/26/2024

🎲

Total Score

0

Sina at FigNews 2024: Multilingual Datasets Annotated with Bias and Propaganda

Lina Duaibes, Areej Jaber, Mustafa Jarrar, Ahmad Qadi, Mais Qandeel

The proliferation of bias and propaganda on social media is an increasingly significant concern, leading to the development of techniques for automatic detection. This article presents a multilingual corpus of 12, 000 Facebook posts fully annotated for bias and propaganda. The corpus was created as part of the FigNews 2024 Shared Task on News Media Narratives for framing the Israeli War on Gaza. It covers various events during the War from October 7, 2023 to January 31, 2024. The corpus comprises 12, 000 posts in five languages (Arabic, Hebrew, English, French, and Hindi), with 2, 400 posts for each language. The annotation process involved 10 graduate students specializing in Law. The Inter-Annotator Agreement (IAA) was used to evaluate the annotations of the corpus, with an average IAA of 80.8% for bias and 70.15% for propaganda annotations. Our team was ranked among the bestperforming teams in both Bias and Propaganda subtasks. The corpus is open-source and available at https://sina.birzeit.edu/fada

Read more

7/15/2024

ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content
Total Score

0

ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content

Maram Hasanain, Md. Arid Hasan, Fatema Ahmed, Reem Suwaileh, Md. Rafiul Biswas, Wajdi Zaghouani, Firoj Alam

We present an overview of the second edition of the ArAIEval shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. In this edition, ArAIEval offers two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. A total of 14 teams participated in the final evaluation phase, with 6 and 9 teams participating in Tasks 1 and 2, respectively. Finally, 11 teams submitted system description papers. Across both tasks, we observed that fine-tuning transformer models such as AraBERT was at the core of the majority of the participating systems. We provide a description of the task setup, including a description of the dataset construction and the evaluation setup. We further provide a brief overview of the participating systems. All datasets and evaluation scripts are released to the research community (https://araieval.gitlab.io/). We hope this will enable further research on these important tasks in Arabic.

Read more

7/8/2024

🔎

Total Score

0

ThatiAR: Subjectivity Detection in Arabic News Sentences

Reem Suwaileh, Maram Hasanain, Fatema Hubail, Wajdi Zaghouani, Firoj Alam

Detecting subjectivity in news sentences is crucial for identifying media bias, enhancing credibility, and combating misinformation by flagging opinion-based content. It provides insights into public sentiment, empowers readers to make informed decisions, and encourages critical thinking. While research has developed methods and systems for this purpose, most efforts have focused on English and other high-resourced languages. In this study, we present the first large dataset for subjectivity detection in Arabic, consisting of ~3.6K manually annotated sentences, and GPT-4o based explanation. In addition, we included instructions (both in English and Arabic) to facilitate LLM based fine-tuning. We provide an in-depth analysis of the dataset, annotation process, and extensive benchmark results, including PLMs and LLMs. Our analysis of the annotation process highlights that annotators were strongly influenced by their political, cultural, and religious backgrounds, especially at the beginning of the annotation process. The experimental results suggest that LLMs with in-context learning provide better performance. We aim to release the dataset and resources for the community.

Read more

6/11/2024