Epistemological Bias As a Means for the Automated Detection of Injustices in Text
0
Sign in to get full access
Overview
- This paper explores the use of epistemological bias as a means for automatically detecting injustices in text.
- The researchers propose a novel approach that leverages biases in knowledge representation to identify instances of epistemic injustice.
- The method aims to uncover biases and inequities in how different types of knowledge and perspectives are valued and represented in textual data.
Plain English Explanation
The paper is about using a concept called "epistemological bias" to automatically find instances of unfairness or injustice in written text. Epistemological bias refers to the biases and inequities in how different types of knowledge and viewpoints are valued and represented.
The researchers developed a new technique that taps into these biases to identify cases where certain knowledge or perspectives are being overlooked or devalued. This could help uncover hidden biases and unfair treatment in things like news articles, social media posts, or other textual data.
The idea is that by understanding the underlying biases in how information is represented, we can build AI systems that are better at detecting when certain voices or viewpoints are being marginalized or dismissed unfairly. This could have important applications in fields like media monitoring or language model evaluation to address issues of epistemic injustice and algorithmic bias.
Technical Explanation
The paper proposes a novel approach for the automated detection of epistemic injustice in text, leveraging insights from the philosophical concept of epistemological bias. The key idea is that the relative valuation and representation of different forms of knowledge can serve as a proxy for identifying instances where certain perspectives or knowledge claims are being marginalized or devalued.
The researchers developed a multi-stage framework that first extracts relevant knowledge claims from the input text. It then assesses the relative "epistemic status" assigned to these claims based on linguistic and contextual cues. By modeling these epistemic biases, the system can flag potential cases of epistemic injustice where certain knowledge is systematically undervalued or excluded.
The approach was evaluated on a dataset of news articles, demonstrating its ability to surface instances of epistemic bias and uncover patterns of unfair treatment across different social groups and knowledge domains. The results highlight the potential of leveraging epistemological analysis for the automated detection and mitigation of unjust knowledge practices in textual data.
Critical Analysis
The paper presents a novel and promising approach for addressing issues of epistemic injustice in textual data. By shifting the focus from individual biases to the systemic biases embedded in how knowledge is represented, the proposed framework offers a more holistic perspective on uncovering unfair treatment.
However, the researchers acknowledge several limitations and caveats. The accuracy of the system is dependent on the quality and comprehensiveness of the underlying ontologies used to model epistemic status. There are also challenges in reliably extracting knowledge claims and inferring their relative valuation from natural language text.
Additionally, the paper does not deeply explore the ethical implications and potential misuses of such a system. There are valid concerns about the risks of automating the detection of epistemic injustice, as this technology could be abused to reinforce existing power structures or suppress alternative viewpoints.
Further research is needed to address these limitations and explore the broader societal impacts. Careful consideration of stakeholder perspectives and the development of appropriate governance frameworks will be crucial to ensure this technology is deployed responsibly and equitably. Ongoing collaboration between technical researchers, domain experts, and affected communities will be essential to guide the responsible development of these capabilities.
Conclusion
This paper presents a novel approach for the automated detection of epistemic injustice in textual data by leveraging insights from the philosophical concept of epistemological bias. The proposed framework offers a promising avenue for uncovering systematic biases in how different forms of knowledge are valued and represented.
While the research shows promising results, it also highlights the need for careful consideration of the ethical implications and potential misuses of such technology. Responsible development and deployment, grounded in close collaboration with stakeholders, will be crucial to ensuring these capabilities are applied in a way that promotes fairness and social justice, rather than entrenching existing inequities.
Overall, this work represents an important step forward in addressing the complex challenge of epistemic injustice, with significant potential implications for fields like media monitoring, language model evaluation, and the broader quest for more equitable and inclusive knowledge practices.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Epistemological Bias As a Means for the Automated Detection of Injustices in Text
Kenya Andrews, Lamogha Chiazor
Injustice occurs when someone experiences unfair treatment or their rights are violated and is often due to the presence of implicit biases and prejudice such as stereotypes. The automated identification of injustice in text has received little attention, due in part to the fact that underlying implicit biases or stereotypes are rarely explicitly stated and that instances often occur unconsciously due to the pervasive nature of prejudice in society. Here, we describe a novel framework that combines the use of a fine-tuned BERT-based bias detection model, two stereotype detection models, and a lexicon-based approach to show that epistemological biases (i.e., words, which presupposes, entails, asserts, hedges, or boosts text to erode or assert a person's capacity as a knower) can assist with the automatic detection of injustice in text. The news media has many instances of injustice (i.e. discriminatory narratives), thus it is our use case here. We conduct and discuss an empirical qualitative research study which shows how the framework can be applied to detect injustices, even at higher volumes of data.
Read more7/9/2024
🤖
0
Epistemic Injustice in Generative AI
Jackie Kay, Atoosa Kasirzadeh, Shakir Mohamed
This paper investigates how generative AI can potentially undermine the integrity of collective knowledge and the processes we rely on to acquire, assess, and trust information, posing a significant threat to our knowledge ecosystem and democratic discourse. Grounded in social and political philosophy, we introduce the concept of emph{generative algorithmic epistemic injustice}. We identify four key dimensions of this phenomenon: amplified and manipulative testimonial injustice, along with hermeneutical ignorance and access injustice. We illustrate each dimension with real-world examples that reveal how generative AI can produce or amplify misinformation, perpetuate representational harm, and create epistemic inequities, particularly in multilingual contexts. By highlighting these injustices, we aim to inform the development of epistemically just generative AI systems, proposing strategies for resistance, system design principles, and two approaches that leverage generative AI to foster a more equitable information ecosystem, thereby safeguarding democratic values and the integrity of knowledge production.
Read more8/22/2024
0
Whose Knowledge is Valued?: Epistemic Injustice in CSCW Applications
Leah Hope Ajmani, Jasmine C Foriest, Jordan Taylor, Kyle Pittman, Sarah Gilbert, Michael Ann Devito
Social computing scholars have long known that people do not interact with knowledge in straightforward ways, especially in digital environments. While policies around knowledge are essential for targeting misinformation, they are value-laden; in choosing how to present information, we undermine non-traditional -- often non-Western -- ways of knowing. Epistemic injustice is the systemic exclusion of certain people and methods from the knowledge canon. Epistemic injustice chips away at one's testimony and vocabulary until they are stripped of their due right to know and understand. In this paper, we articulate how epistemic injustice in sociotechnical applications leads to material harm. Inspired by a hybrid collaborative autoethnography of 14 CSCW practitioners, we present three cases of epistemic injustice in sociotechnical applications: online transgender healthcare, identity sensemaking on r/bisexual, and Indigenous ways of knowing on r/AskHistorians. We further explore signature tensions across our autoethnographic materials and relate them to previous CSCW research areas and personal non-technological experiences. We argue that epistemic injustice can serve as a unifying and intersectional lens for CSCW research by surfacing dimensions of epistemic community and power. Finally, we present a call to action of three changes the CSCW community should make to move toward its own goals of research justice. We call for CSCW researchers to center individual experiences, bolster communities, and remediate issues of epistemic power as a means towards epistemic justice. In sum, we recount, synthesize, and propose solutions for the various forms of epistemic injustice that CSCW sites of study -- including CSCW itself -- propagate.
Read more7/8/2024
👁️
0
Aleatoric and Epistemic Discrimination: Fundamental Limits of Fairness Interventions
Hao Wang, Luxi He, Rui Gao, Flavio P. Calmon
Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric discrimination, which is inherent in the data distribution, and epistemic discrimination, which is due to decisions made during model development. We quantify aleatoric discrimination by determining the performance limits of a model under fairness constraints, assuming perfect knowledge of the data distribution. We demonstrate how to characterize aleatoric discrimination by applying Blackwell's results on comparing statistical experiments. We then quantify epistemic discrimination as the gap between a model's accuracy when fairness constraints are applied and the limit posed by aleatoric discrimination. We apply this approach to benchmark existing fairness interventions and investigate fairness risks in data with missing values. Our results indicate that state-of-the-art fairness interventions are effective at removing epistemic discrimination on standard (overused) tabular datasets. However, when data has missing values, there is still significant room for improvement in handling aleatoric discrimination.
Read more4/17/2024