Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities
2406.01757
0
0
👀
Abstract
The rise of foundation models holds immense promise for advancing AI, but this progress may amplify existing risks and inequalities, leaving marginalized communities behind. In this position paper, we discuss that disparities towards marginalized communities - performance, representation, privacy, robustness, interpretability and safety - are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We contrast foundation models with traditional models and highlight the potential for exacerbated disparity against marginalized communities. Moreover, we emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences, specifically to the people on the margin. We define marginalized communities within the machine learning context and explore the multifaceted nature of disparities. We analyze the sources of these disparities, tracing them from data creation, training and deployment procedures to highlight the complex technical and socio-technical landscape. To mitigate the pressing crisis, we conclude with a set of calls to action to mitigate disparity at its source.
Create account to get full access
Overview
- The paper examines the issue of cascading disparity towards marginalized communities, particularly in the context of technology and AI.
- It explores how systemic biases and inequalities can become amplified and perpetuated through the development and deployment of AI systems.
- The research aims to uncover the mechanisms behind this "cracking the code of cascading disparity" and identify ways to address the problem.
Plain English Explanation
The paper looks at how the use of technology, especially artificial intelligence (AI), can actually make things worse for marginalized communities instead of better. It explains how the biases and unfairness already present in society can get built into the AI systems that are meant to help, and then get amplified and spread even further.
The researchers try to understand how this "cascading disparity" happens, where the problems just keep getting worse and worse for already disadvantaged groups. They want to find ways to break this cycle and make sure that technology and AI are actually used to support and empower marginalized communities, rather than continuing to hold them back.
Technical Explanation
The paper explores the concept of "cascading disparity," where systemic biases and inequalities become entrenched and amplified through the development and deployment of AI systems. The authors argue that marginalized communities often face compounded disadvantages as a result of these dynamics.
The research examines the underlying mechanisms behind this phenomenon, drawing on insights from fields such as Exploiting Margin: How Capitalism Fuels AI at the Expense of Marginalized Communities, Reducing Barriers to Entry for Foundation Model Training, and Disparisk: Assessing and Interpreting Disparity Risks of Datasets. The authors also consider the role of Participation: The Age of Foundation Models and Mind the Gap: A Causal Perspective on Bias Amplification in Prediction in shaping these dynamics.
Critical Analysis
The paper provides a nuanced and thought-provoking analysis of the complex interplay between technology, AI, and societal inequalities. While the authors make a compelling case for the dangers of cascading disparity, they also acknowledge the limitations of their research and the need for further investigation.
One potential area for concern is the challenge of addressing deeply entrenched systemic biases, which may require fundamental changes to the underlying social, economic, and political structures. The paper does not delve deeply into these broader systemic issues, focusing more on the technological aspects.
Additionally, the researchers note that the specific mechanisms underlying cascading disparity may vary across different contexts and communities, necessitating more tailored and context-sensitive approaches to mitigation. Further research exploring the nuances of these dynamics would be valuable.
Conclusion
This paper makes a significant contribution to the ongoing discourse around the responsible development and deployment of AI systems. By highlighting the issue of cascading disparity, the authors underscore the urgent need to address the complex intersections of technology, power, and marginalization.
The insights provided in this research can inform policy, practice, and further academic study, as stakeholders work to ensure that technological advancements, including AI, serve to empower and uplift marginalized communities rather than perpetuating and amplifying existing inequalities.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Participation in the age of foundation models
Harini Suresh, Emily Tseng, Meg Young, Mary L. Gray, Emma Pierson, Karen Levy
0
0
Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of public services. Alongside these opportunities is the risk that these systems reify existing power imbalances and cause disproportionate harm to marginalized communities. Participatory approaches hold promise to instead lend agency and decision-making power to marginalized stakeholders. But existing approaches in participatory AI/ML are typically deeply grounded in context - how do we apply these approaches to foundation models, which are, by design, disconnected from context? Our paper interrogates this question. First, we examine existing attempts at incorporating participation into foundation models. We highlight the tension between participation and scale, demonstrating that it is intractable for impacted communities to meaningfully shape a foundation model that is intended to be universally applicable. In response, we develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation. In addition to the foundation layer, our framework proposes the subfloor'' layer, in which stakeholders develop shared technical infrastructure, norms and governance for a grounded domain, and the surface'' layer, in which affected communities shape the use of a foundation model for a specific downstream task. The intermediate subfloor'' layer scopes the range of potential harms to consider, and affords communities more concrete avenues for deliberation and intervention. At the same time, it avoids duplicative effort by scaling input across relevant use cases. Through three case studies in clinical care, financial services, and journalism, we illustrate how this multi-layer model can create more meaningful opportunities for participation than solely intervening at the foundation layer.
5/31/2024
🤖
Exploiting the Margin: How Capitalism Fuels AI at the Expense of Minoritized Groups
Nelson Col'on Vargas
0
0
This paper explores the intricate relationship between capitalism, racial injustice, and artificial intelligence (AI), arguing that AI acts as a contemporary vehicle for age-old forms of exploitation. By linking historical patterns of racial and economic oppression with current AI practices, this study illustrates how modern technology perpetuates and deepens societal inequalities. It specifically examines how AI is implicated in the exploitation of marginalized communities through underpaid labor in the gig economy, the perpetuation of biases in algorithmic decision-making, and the reinforcement of systemic barriers that prevent these groups from benefiting equitably from technological advances. Furthermore, the paper discusses the role of AI in extending and intensifying the social, economic, and psychological burdens faced by these communities, highlighting the problematic use of AI in surveillance, law enforcement, and mental health contexts. The analysis concludes with a call for transformative changes in how AI is developed and deployed. Advocating for a reevaluation of the values driving AI innovation, the paper promotes an approach that integrates social justice and equity into the core of technological design and policy. This shift is crucial for ensuring that AI serves as a tool for societal improvement, fostering empowerment and healing rather than deepening existing divides.
5/2/2024
Reducing the Barriers to Entry for Foundation Model Training
Paolo Faraboschi, Ellis Giles, Justin Hotard, Konstanty Owczarek, Andrew Wheeler
0
0
The world has recently witnessed an unprecedented acceleration in demands for Machine Learning and Artificial Intelligence applications. This spike in demand has imposed tremendous strain on the underlying technology stack in supply chain, GPU-accelerated hardware, software, datacenter power density, and energy consumption. If left on the current technological trajectory, future demands show insurmountable spending trends, further limiting market players, stifling innovation, and widening the technology gap. To address these challenges, we propose a fundamental change in the AI training infrastructure throughout the technology ecosystem. The changes require advancements in supercomputing and novel AI training approaches, from high-end software to low-level hardware, microprocessor, and chip design, while advancing the energy efficiency required by a sustainable infrastructure. This paper presents the analytical framework that quantitatively highlights the challenges and points to the opportunities to reduce the barriers to entry for training large language models.
4/16/2024
🔗
DispaRisk: Assessing and Interpreting Disparity Risks in Datasets
Jonathan Vasquez, Carlotta Domeniconi, Huzefa Rangwala
0
0
Machine Learning algorithms (ML) impact virtually every aspect of human lives and have found use across diverse sectors, including healthcare, finance, and education. Often, ML algorithms have been found to exacerbate societal biases presented in datasets, leading to adversarial impacts on subsets/groups of individuals, in many cases minority groups. To effectively mitigate these untoward effects, it is crucial that disparities/biases are identified and assessed early in a ML pipeline. This proactive approach facilitates timely interventions to prevent bias amplification and reduce complexity at later stages of model development. In this paper, we introduce DispaRisk, a novel framework designed to proactively assess the potential risks of disparities in datasets during the initial stages of the ML pipeline. We evaluate DispaRisk's effectiveness by benchmarking it with commonly used datasets in fairness research. Our findings demonstrate the capabilities of DispaRisk to identify datasets with a high-risk of discrimination, model families prone to biases, and characteristics that heighten discrimination susceptibility in a ML pipeline. The code for our experiments is available in the following repository: https://github.com/jovasque156/disparisk
5/22/2024