Participation in the age of foundation models
2405.19479
0
0
Abstract
Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of public services. Alongside these opportunities is the risk that these systems reify existing power imbalances and cause disproportionate harm to marginalized communities. Participatory approaches hold promise to instead lend agency and decision-making power to marginalized stakeholders. But existing approaches in participatory AI/ML are typically deeply grounded in context - how do we apply these approaches to foundation models, which are, by design, disconnected from context? Our paper interrogates this question. First, we examine existing attempts at incorporating participation into foundation models. We highlight the tension between participation and scale, demonstrating that it is intractable for impacted communities to meaningfully shape a foundation model that is intended to be universally applicable. In response, we develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation. In addition to the foundation layer, our framework proposes the subfloor'' layer, in which stakeholders develop shared technical infrastructure, norms and governance for a grounded domain, and the surface'' layer, in which affected communities shape the use of a foundation model for a specific downstream task. The intermediate subfloor'' layer scopes the range of potential harms to consider, and affords communities more concrete avenues for deliberation and intervention. At the same time, it avoids duplicative effort by scaling input across relevant use cases. Through three case studies in clinical care, financial services, and journalism, we illustrate how this multi-layer model can create more meaningful opportunities for participation than solely intervening at the foundation layer.
Create account to get full access
Overview
- This paper explores the challenges and opportunities for public participation in the age of foundation models, which are large, pre-trained machine learning models that can be adapted to a wide range of tasks.
- The authors discuss the importance of engaging diverse stakeholders, including communities, in the development and governance of foundation models to ensure they are used responsibly and for the benefit of society.
- The paper highlights the need for new frameworks and approaches to enable meaningful public participation in the foundation model ecosystem.
Plain English Explanation
Foundation models are powerful AI systems that can be adapted for many different tasks, from language processing to image generation. As these models become more prevalent, it's crucial that the public has a voice in how they are developed and used.
The authors of this paper argue that foundation models shouldn't be created in a vacuum by tech companies and researchers alone. Instead, they need to involve a wide range of stakeholders, including the communities that may be impacted by these models. This could help ensure that foundation models are designed and deployed responsibly, with safeguards in place to prevent misuse or harm.
The paper explores some of the barriers to public participation, such as the technical complexity of foundation models and the lack of clear governance frameworks. It also highlights examples of initiatives that are trying to bridge this gap, like Towards a Framework for Openness in Foundation Models and Towards Responsible and Safe AI in the Era of Foundation Models.
Overall, the paper emphasizes the importance of empowering diverse voices and perspectives in the foundation model ecosystem. This could help ensure these powerful technologies are developed and used in ways that benefit everyone, not just the tech elite.
Technical Explanation
The paper begins by highlighting the rapid progress of foundation models, which are large, pre-trained AI models that can be adapted to a wide range of tasks. The authors note that as these models become more ubiquitous, it's crucial to address the challenge of public participation in their development and governance.
The authors review related work in this area, including efforts to promote openness and transparency in foundation models and develop responsible and safe approaches to foundation model development. They also discuss how foundation models are being applied in domains like education and healthcare, highlighting the need for public input in these areas.
The core of the paper explores the challenges and opportunities for public participation in the foundation model ecosystem. The authors identify barriers such as the technical complexity of these models, the lack of clear governance frameworks, and the potential for marginalization of certain communities. They also discuss initiatives that are attempting to address these issues, such as efforts to ensure data authenticity, consent, and provenance in AI systems.
Overall, the paper argues that meaningful public participation is essential to ensure foundation models are developed and deployed responsibly, with the interests of diverse stakeholders in mind. The authors call for new frameworks and approaches to enable this participation, recognizing the significant technical and social challenges involved.
Critical Analysis
The paper raises important concerns about the need for public participation in the development and governance of foundation models, which are becoming increasingly prevalent and influential. The authors make a compelling case that these powerful AI systems should not be created in isolation by tech companies and researchers, but rather should involve input and oversight from a wide range of stakeholders, including affected communities.
One potential limitation of the paper is that it focuses primarily on the conceptual and institutional challenges of enabling public participation, without delving deeply into specific proposals or case studies. While the authors do highlight some relevant initiatives, a more detailed exploration of these efforts and their successes or shortcomings could have strengthened the analysis.
Additionally, the paper could have more explicitly addressed potential tensions or tradeoffs that may arise between the goals of public participation and other priorities, such as technological innovation or commercial interests. A more nuanced discussion of these competing considerations could have added depth to the critical analysis.
Nevertheless, the paper makes a valuable contribution by elevating the importance of public participation in the foundation model ecosystem. As these powerful AI systems continue to proliferate, it will be crucial for policymakers, researchers, and tech companies to work together to develop robust frameworks and mechanisms for meaningful public engagement and oversight.
Conclusion
This paper underscores the critical need for public participation in the development and governance of foundation models, which are becoming increasingly ubiquitous and influential. The authors argue that these powerful AI systems should not be created in isolation by tech companies and researchers, but rather should involve input and oversight from a wide range of stakeholders, including affected communities.
The paper highlights the significant technical and social challenges involved in enabling meaningful public participation, but also points to emerging initiatives that are attempting to address these issues. As foundation models continue to advance and be deployed in diverse domains, the authors call for the development of new frameworks and approaches to ensure these technologies are developed and used responsibly, with the interests of all stakeholders in mind.
Overall, this paper makes a compelling case for the importance of public participation in the foundation model ecosystem, and serves as a valuable contribution to the ongoing discussions around the responsible development and governance of powerful AI systems.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🎲
Towards a Framework for Openness in Foundation Models: Proceedings from the Columbia Convening on Openness in Artificial Intelligence
Adrien Basdevant, Camille Franc{c}ois, Victor Storchan, Kevin Bankston, Ayah Bdeir, Brian Behlendorf, Merouane Debbah, Sayash Kapoor, Yann LeCun, Mark Surman, Helen King-Turvey, Nathan Lambert, Stefano Maffulli, Nik Marda, Govind Shivkumar, Justine Tunney
0
0
Over the past year, there has been a robust debate about the benefits and risks of open sourcing foundation models. However, this discussion has often taken place at a high level of generality or with a narrow focus on specific technical attributes. In part, this is because defining open source for foundation models has proven tricky, given its significant differences from traditional software development. In order to inform more practical and nuanced decisions about opening AI systems, including foundation models, this paper presents a framework for grappling with openness across the AI stack. It summarizes previous work on this topic, analyzes the various potential reasons to pursue openness, and outlines how openness varies in different parts of the AI stack, both at the model and at the system level. In doing so, its authors hope to provide a common descriptive framework to deepen a nuanced and rigorous understanding of openness in AI and enable further work around definitions of openness and safety in AI.
5/28/2024
🤖
Towards Responsible and Safe AI in the Era of Foudnation Models: A Reference Architecture for Designing Foundation Model based Systems
Qinghua Lu, Liming Zhu, Xiwei Xu, Zhenchang Xing, Jon Whittle
0
0
The release of ChatGPT, Gemini, and other large language model has drawn huge interests on foundations models. There is a broad consensus that foundations models will be the fundamental building blocks for future AI systems. However, there is a lack of systematic guidance on the architecture design. Particularly, the the rapidly growing capabilities of foundations models can eventually absorb other components of AI systems, posing challenges of moving boundary and interface evolution in architecture design. Furthermore, incorporating foundations models into AI systems raises significant concerns about responsible and safe AI due to their opaque nature and rapidly advancing intelligence. To address these challenges, the paper first presents an architecture evolution of AI systems in the era of foundation models, transitioning from foundation-model-as-a-connector to foundation-model-as-a-monolithic architecture. The paper then identifies key design decisions and proposes a pattern-oriented reference architecture for designing responsible foundation-model-based systems. The patterns can enable the potential of foundation models while ensuring associated risks.
5/28/2024
👀
Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities
Golnoosh Farnadi, Mohammad Havaei, Negar Rostamzadeh
0
0
The rise of foundation models holds immense promise for advancing AI, but this progress may amplify existing risks and inequalities, leaving marginalized communities behind. In this position paper, we discuss that disparities towards marginalized communities - performance, representation, privacy, robustness, interpretability and safety - are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We contrast foundation models with traditional models and highlight the potential for exacerbated disparity against marginalized communities. Moreover, we emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences, specifically to the people on the margin. We define marginalized communities within the machine learning context and explore the multifaceted nature of disparities. We analyze the sources of these disparities, tracing them from data creation, training and deployment procedures to highlight the complex technical and socio-technical landscape. To mitigate the pressing crisis, we conclude with a set of calls to action to mitigate disparity at its source.
6/5/2024
⚙️
Foundation Models for Education: Promises and Prospects
Tianlong Xu, Richard Tong, Jing Liang, Xing Fan, Haoyang Li, Qingsong Wen
0
0
With the advent of foundation models like ChatGPT, educators are excited about the transformative role that AI might play in propelling the next education revolution. The developing speed and the profound impact of foundation models in various industries force us to think deeply about the changes they will make to education, a domain that is critically important for the future of humans. In this paper, we discuss the strengths of foundation models, such as personalized learning, education inequality, and reasoning capabilities, as well as the development of agent architecture tailored for education, which integrates AI agents with pedagogical frameworks to create adaptive learning environments. Furthermore, we highlight the risks and opportunities of AI overreliance and creativity. Lastly, we envision a future where foundation models in education harmonize human and AI capabilities, fostering a dynamic, inclusive, and adaptive educational ecosystem.
5/21/2024