Autonomation, not Automation: Activities and Needs of Fact-checkers as a Basis for Designing Human-Centered AI Systems
0
🤖
Sign in to get full access
Overview
- This study explores the needs and challenges faced by fact-checkers in non-English speaking regions of Europe.
- The researchers conducted in-depth interviews with fact-checkers in Central Europe and followed up with a survey of European fact-checkers.
- The goal was to better understand the fact-checking process and identify opportunities for AI systems to assist fact-checkers.
Plain English Explanation
The researchers wanted to find better ways to use AI models to help fact-checkers do their jobs more effectively. Fact-checkers are people who investigate claims and information to determine if they are true or false. The researchers found that fact-checkers in non-English speaking parts of Europe face some unique challenges that aren't well understood.
To learn more, the researchers interviewed fact-checkers in Central Europe and then surveyed fact-checkers across Europe. They wanted to understand the day-to-day work of fact-checking and identify any problems or needs that could be addressed by AI systems.
The researchers discovered that fact-checkers have a wide range of activities and face various challenges, such as dealing with misinformation in multiple languages. By understanding these issues, the researchers were able to suggest new ways that AI could assist fact-checkers and improve the overall fact-checking process.
Technical Explanation
The researchers conducted semi-structured interviews with fact-checkers in Central Europe to understand their activities and the problems they face. They used iterative content analysis to analyze the interview data and identify the most significant issues.
To validate these findings, the researchers then conducted a survey of European fact-checkers, collecting 24 responses from 20 countries, representing 62% of active European signatories of the International Fact-Checking Network (IFCN).
By aligning the insights from the interviews and survey with prior research, the researchers were able to create conceptual models that help explain the fact-checking process. They also identified three additional stages that could be incorporated into the fact-checking process for AI research.
Additionally, the researchers mapped the fact-checkers' activities and needs to relevant tasks for AI research, highlighting new opportunities for AI researchers and developers to focus their efforts in this domain.
Critical Analysis
The researchers provide a valuable contribution by examining the fact-checking process in non-English speaking regions, which has been largely unexplored in prior research. Their findings reveal the variability and unique challenges faced by fact-checkers in these contexts, underscoring the importance of understanding the diverse needs of stakeholders.
However, the study is limited by its relatively small sample size, particularly for the survey component. While the researchers were able to collect responses from a significant portion of active European IFCN signatories, a larger and more diverse sample could help strengthen the generalizability of the findings.
Additionally, the researchers acknowledge that their conceptual models and proposed extensions to the fact-checking process would benefit from further validation and refinement through additional research and collaboration with fact-checkers and AI experts.
Conclusion
This study offers important insights into the needs and challenges faced by fact-checkers, particularly in non-English speaking regions of Europe. By understanding the variability of fact-checking work and the specific problems encountered by these stakeholders, the researchers have identified new opportunities for AI systems to assist in the fact-checking process.
The findings from this research can help guide the focus of AI research in this domain, leading to the development of more effective and tailored solutions that better meet the needs of fact-checkers. This, in turn, could contribute to more efficient and reliable fact-checking, ultimately helping to mitigate the negative impacts of false information.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🤖
0
Autonomation, not Automation: Activities and Needs of Fact-checkers as a Basis for Designing Human-Centered AI Systems
Andrea Hrckova, Robert Moro, Ivan Srba, Jakub Simko, Maria Bielikova
To mitigate the negative effects of false information more effectively, the development of Artificial Intelligence (AI) systems assisting fact-checkers is needed. Nevertheless, the lack of focus on the needs of these stakeholders results in their limited acceptance and skepticism toward automating the whole fact-checking process. In this study, we conducted semi-structured in-depth interviews with Central European fact-checkers. Their activities and problems were analyzed using iterative content analysis. The most significant problems were validated with a survey of European fact-checkers, in which we collected 24 responses from 20 countries, i.e., 62% of active European signatories of the International Fact-Checking Network (IFCN). Our contributions include an in-depth examination of the variability of fact-checking work in non-English speaking regions, which still remained largely uncovered. By aligning them with the knowledge from prior studies, we created conceptual models that help understand the fact-checking processes. Thanks to the interdisciplinary collaboration, we extend the fact-checking process in AI research by three additional stages. In addition, we mapped our findings on the fact-checkers' activities and needs to the relevant tasks for AI research. The new opportunities identified for AI researchers and developers have implications for the focus of AI research in this domain.
Read more8/14/2024
📶
0
XAI in Automated Fact-Checking? The Benefits Are Modest and There's No One-Explanation-Fits-All
Gionnieve Lim, Simon T. Perrault
The massive volume of online information along with the issue of misinformation has spurred active research in the automation of fact-checking. Like fact-checking by human experts, it is not enough for an automated fact-checker to just be accurate, but also be able to inform and convince the user of the validity of its predictions. This becomes viable with explainable artificial intelligence (XAI). In this work, we conduct a study of XAI fact-checkers involving 180 participants to determine how users' actions towards news and their attitudes towards explanations are affected by the XAI. Our results suggest that XAI has limited effects on users' agreement with the veracity prediction of the automated fact-checker and on their intent to share news. However, XAI nudges users towards forming uniform judgments of news veracity, thereby signaling their reliance on the explanations. We also found polarizing preferences towards XAI and raise several design considerations on them.
Read more6/21/2024
0
The Impact and Opportunities of Generative AI in Fact-Checking
Robert Wolfe, Tanushree Mitra
Generative AI appears poised to transform white collar professions, with more than 90% of Fortune 500 companies using OpenAI's flagship GPT models, which have been characterized as general purpose technologies capable of effecting epochal changes in the economy. But how will such technologies impact organizations whose job is to verify and report factual information, and to ensure the health of the information ecosystem? To investigate this question, we conducted 30 interviews with N=38 participants working at 29 fact-checking organizations across six continents, asking about how they use generative AI and the opportunities and challenges they see in the technology. We found that uses of generative AI envisioned by fact-checkers differ based on organizational infrastructure, with applications for quality assurance in Editing, for trend analysis in Investigation, and for information literacy in Advocacy. We used the TOE framework to describe participant concerns ranging from the Technological (lack of transparency), to the Organizational (resource constraints), to the Environmental (uncertain and evolving policy). Building on the insights of our participants, we describe value tensions between fact-checking and generative AI, and propose a novel Verification dimension to the design space of generative models for information verification work. Finally, we outline an agenda for fairness, accountability, and transparency research to support the responsible use of generative AI in fact-checking. Throughout, we highlight the importance of human infrastructure and labor in producing verified information in collaboration with AI. We expect that this work will inform not only the scientific literature on fact-checking, but also contribute to understanding of organizational adaptation to a powerful but unreliable new technology.
Read more5/28/2024
0
Automated Justification Production for Claim Veracity in Fact Checking: A Survey on Architectures and Approaches
Islam Eldifrawi, Shengrui Wang, Amine Trabelsi
Automated Fact-Checking (AFC) is the automated verification of claim accuracy. AFC is crucial in discerning truth from misinformation, especially given the huge amounts of content are generated online daily. Current research focuses on predicting claim veracity through metadata analysis and language scrutiny, with an emphasis on justifying verdicts. This paper surveys recent methodologies, proposing a comprehensive taxonomy and presenting the evolution of research in that landscape. A comparative analysis of methodologies and future directions for improving fact-checking explainability are also discussed.
Read more7/19/2024