Sentence-level Media Bias Analysis with Event Relation Graph

Read original: arXiv:2404.01722 - Published 4/3/2024 by Yuanyuan Lei, Ruihong Huang
Total Score

0

Sentence-level Media Bias Analysis with Event Relation Graph

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • The paper presents a new approach for analyzing media bias at the sentence level using an "Event Relation Graph"
  • This method aims to provide a more granular and nuanced understanding of bias compared to existing approaches that focus on document-level analysis
  • The authors develop a neural network model to extract event relations from text and use these to build a graph-based representation of the content
  • They then use this graph structure to identify biased statements and quantify the degree of bias present

Plain English Explanation

The paper tackles the challenge of identifying media bias, which occurs when news coverage presents a slanted or one-sided perspective on events and issues. Existing methods for detecting bias often look at an entire article or document, but the authors argue this can miss important nuances at the sentence level.

Their approach involves breaking down text into individual events - things like who did what to whom, when and where. These event relationships are then used to construct a graph-like structure that captures the overall narrative and flow of information. The authors' neural network model is able to automatically extract these event-level details from text.

By analyzing the patterns and connections within this event relation graph, the researchers can identify sentences that express bias, whether by downplaying certain facts, emphasizing others, or framing events in a particular way. This allows for a more granular and contextual assessment of bias compared to simply looking at word choices or the overall tone of a document.

The key innovation is using this structured event-based representation to get beneath the surface and really understand how bias manifests in the language and storytelling choices of media outlets. This could have important applications in media monitoring, journalistic integrity, and public discourse.

Technical Explanation

The paper introduces a novel "Event Relation Graph" (ERG) approach for sentence-level media bias analysis. The key steps are:

  1. Event Extraction: A neural network model is used to automatically extract event tuples (subject, verb, object, time, location) from input text. This breaks down the content into structured event-level representations.

  2. Event Relation Graph Construction: The extracted events are used to build a graph structure, where nodes represent events and edges capture logical/causal relations between them. This models the flow and framing of information in the text.

  3. Bias Identification: Graph-based features are then used to identify biased statements, such as events that are downplayed, emphasized, or framed in a particular way compared to the overall narrative.

The authors evaluate their approach on a dataset of news articles and find that it outperforms existing bias detection methods that operate at the document level. The ERG model is able to pinpoint specific sentences exhibiting bias, as well as quantify the degree of bias present.

Critical Analysis

A key strength of the ERG approach is its ability to go beyond simple word-level analysis and capture more complex narrative structures and causal relationships within text. This aligns with the authors' argument that bias often manifests in subtle ways in how events are connected and framed, rather than just lexical choices.

That said, the paper does not extensively discuss limitations or potential issues with the method. For instance, the event extraction model may struggle with nuanced, ambiguous, or implicit event descriptions that are common in natural language. There could also be challenges in generalizing the approach to diverse domains and styles of text beyond news articles.

Additionally, while the authors demonstrate the ERG model's effectiveness at detecting bias, they do not delve into the question of what constitutes "true" or "objective" framing of events. Bias is inherently subjective, and reasonable people may disagree on where the line should be drawn. Further research is needed to establish more robust and generalizable standards for assessing bias.

Overall, the paper presents a compelling technical advance in media bias analysis, but there remain open questions around the practical deployment and broader implications of such techniques. Continued exploration and validation will be important as this line of research progresses.

Conclusion

This paper introduces a novel Event Relation Graph approach that can analyze media bias at a granular, sentence-level rather than relying on coarser document-level signals. By extracting structured event representations and modeling their logical connections, the method provides a more nuanced way to identify instances of bias in how information is framed and emphasized.

The technical results demonstrate the effectiveness of this approach compared to existing bias detection techniques. More broadly, this work highlights the potential for sophisticated natural language processing to shed light on the complex dynamics of bias in media and public discourse. As concerns around misinformation and polarization continue to grow, tools like the ERG model could play an important role in promoting transparency and accountability.

However, the research also points to the need for deeper examination of what constitutes objective, unbiased reporting, and how to operationalize such standards. Continued development and real-world application of these methods will require grappling with the inherent subjectivity and context-dependence of bias.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

Sentence-level Media Bias Analysis with Event Relation Graph
Total Score

0

Sentence-level Media Bias Analysis with Event Relation Graph

Yuanyuan Lei, Ruihong Huang

Media outlets are becoming more partisan and polarized nowadays. In this paper, we identify media bias at the sentence level, and pinpoint bias sentences that intend to sway readers' opinions. As bias sentences are often expressed in a neutral and factual way, considering broader context outside a sentence can help reveal the bias. In particular, we observe that events in a bias sentence need to be understood in associations with other events in the document. Therefore, we propose to construct an event relation graph to explicitly reason about event-event relations for sentence-level bias identification. The designed event relation graph consists of events as nodes and four common types of event relations: coreference, temporal, causal, and subevent relations. Then, we incorporate event relation graph for bias sentences identification in two steps: an event-aware language model is built to inject the events and event relations knowledge into the basic language model via soft labels; further, a relation-aware graph attention network is designed to update sentence embedding with events and event relations information based on hard labels. Experiments on two benchmark datasets demonstrate that our approach with the aid of event relation graph improves both precision and recall of bias sentence identification.

Read more

4/3/2024

⚙️

Total Score

0

New!Predicting Sentence-Level Factuality of News and Bias of Media Outlets

Francielle Vargas, Kokil Jaidka, Thiago A. S. Pardo, Fabr'icio Benevenuto

Automated news credibility and fact-checking at scale require accurately predicting news factuality and media bias. This paper introduces a large sentence-level dataset, titled FactNews, composed of 6,191 sentences expertly annotated according to factuality and media bias definitions proposed by AllSides. We use FactNews to assess the overall reliability of news sources, by formulating two text classification problems for predicting sentence-level factuality of news reporting and bias of media outlets. Our experiments demonstrate that biased sentences present a higher number of words compared to factual sentences, besides having a predominance of emotions. Hence, the fine-grained analysis of subjectivity and impartiality of news articles provided promising results for predicting the reliability of media outlets. Finally, due to the severity of fake news and political polarization in Brazil, and the lack of research for Portuguese, both dataset and baseline were proposed for Brazilian Portuguese.

Read more

9/16/2024

💬

Total Score

1

Improving Large Language Models in Event Relation Logical Prediction

Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang, Dongsheng Li

Event relations are crucial for narrative understanding and reasoning. Governed by nuanced logic, event relation extraction (ERE) is a challenging task that demands thorough semantic understanding and rigorous logical reasoning. In this paper, we conduct an in-depth investigation to systematically explore the capability of LLMs in understanding and applying event relation logic. More in detail, we first investigate the deficiencies of LLMs in logical reasoning across different tasks. Our study reveals that LLMs are not logically consistent reasoners, which results in their suboptimal performance on tasks that need rigorous reasoning. To address this, we explore three different approaches to endow LLMs with event relation logic, and thus enable them to generate more coherent answers across various scenarios. Based on our approach, we also contribute a synthesized dataset (LLM-ERL) involving high-order reasoning for evaluation and fine-tuning. Extensive quantitative and qualitative analyses on different tasks also validate the effectiveness of our approaches and provide insights for solving practical tasks with LLMs in future work. Codes are available at https://github.com/chenmeiqii/Teach-LLM-LR.

Read more

8/12/2024

Enhancing Event Reasoning in Large Language Models through Instruction Fine-Tuning with Semantic Causal Graphs
Total Score

0

Enhancing Event Reasoning in Large Language Models through Instruction Fine-Tuning with Semantic Causal Graphs

Mazal Bethany, Emet Bethany, Brandon Wherry, Cho-Yu Chiang, Nishant Vishwamitra, Anthony Rios, Peyman Najafirad

Event detection and text reasoning have become critical applications across various domains. While LLMs have recently demonstrated impressive progress in reasoning abilities, they often struggle with event detection, particularly due to the absence of training methods that consider causal relationships between event triggers and types. To address this challenge, we propose a novel approach for instruction fine-tuning LLMs for event detection. Our method introduces Semantic Causal Graphs (SCGs) to capture both causal relationships and contextual information within text. Building off of SCGs, we propose SCG Instructions for fine-tuning LLMs by focusing on event triggers and their relationships to event types, and employ Low-Rank Adaptation (LoRA) to help preserve the general reasoning abilities of LLMs. Our evaluations demonstrate that training LLMs with SCG Instructions outperforms standard instruction fine-tuning by an average of 35.69% on Event Trigger Classification. Notably, our fine-tuned Mistral 7B model also outperforms GPT-4 on key event detection metrics by an average of 31.01% on Event Trigger Identification, 37.40% on Event Trigger Classification, and 16.43% on Event Classification. We analyze the retention of general capabilities, observing only a minimal average drop of 2.03 points across six benchmarks. This comprehensive study investigates multiple LLMs for the event detection task across various datasets, prompting strategies, and training approaches.

Read more

9/4/2024