Developing trustworthy AI applications with foundation models
2405.04937
0
0
🤖
Abstract
The trustworthiness of AI applications has been the subject of recent research and is also addressed in the EU's recently adopted AI Regulation. The currently emerging foundation models in the field of text, speech and image processing offer completely new possibilities for developing AI applications. This whitepaper shows how the trustworthiness of an AI application developed with foundation models can be evaluated and ensured. For this purpose, the application-specific, risk-based approach for testing and ensuring the trustworthiness of AI applications, as developed in the 'AI Assessment Catalog - Guideline for Trustworthy Artificial Intelligence' by Fraunhofer IAIS, is transferred to the context of foundation models. Special consideration is given to the fact that specific risks of foundation models can have an impact on the AI application and must also be taken into account when checking trustworthiness. Chapter 1 of the white paper explains the fundamental relationship between foundation models and AI applications based on them in terms of trustworthiness. Chapter 2 provides an introduction to the technical construction of foundation models and Chapter 3 shows how AI applications can be developed based on them. Chapter 4 provides an overview of the resulting risks regarding trustworthiness. Chapter 5 shows which requirements for AI applications and foundation models are to be expected according to the draft of the European Union's AI Regulation and Chapter 6 finally shows the system and procedure for meeting trustworthiness requirements.
Get summaries of the top AI research delivered straight to your inbox:
Overview
- This whitepaper explores how to evaluate and ensure the trustworthiness of AI applications developed using foundation models in text, speech, and image processing.
- It applies the risk-based approach for testing and ensuring trustworthy AI, as outlined in the "AI Assessment Catalog - Guideline for Trustworthy Artificial Intelligence" by Fraunhofer IAIS, to the context of foundation models.
- The paper examines the relationship between foundation models and AI applications, the technical construction of foundation models, the development of AI apps using foundation models, the resulting risks to trustworthiness, and how to meet trustworthiness requirements based on the EU's draft AI Regulation.
Plain English Explanation
Foundation models are a new type of artificial intelligence (AI) system that can be used to build a wide variety of AI applications, like chatbots, image generators, and language translators. These foundation models offer exciting possibilities, but they also come with some risks that need to be carefully managed.
This whitepaper explains how to evaluate and ensure the trustworthiness of AI applications that are built using foundation models. It takes the approach developed by Fraunhofer IAIS for testing and ensuring trustworthy AI, and applies it specifically to the context of foundation models.
The paper starts by explaining the relationship between foundation models and the AI apps built on top of them. It then dives into the technical details of how foundation models work. Next, it shows how AI applications can be developed using foundation models. After that, it outlines the various risks to trustworthiness that can arise from using foundation models.
The whitepaper also looks at the upcoming EU AI Regulation and what requirements it will place on AI applications and foundation models. Finally, it describes a process for meeting those trustworthiness requirements.
The goal is to give AI developers and users a clear understanding of how to build and evaluate trustworthy AI applications that are powered by foundation models. By proactively addressing these trustworthiness concerns, we can unlock the full potential of this exciting new AI technology while ensuring it is used responsibly.
Technical Explanation
The paper first explains the fundamental relationship between foundation models and the AI applications built using them. Foundation models are large, general-purpose AI models that can be fine-tuned for a variety of downstream tasks. They offer significant advantages over training AI systems from scratch, but also introduce new risks that must be managed.
Chapter 2 provides an overview of the technical construction of foundation models. These models are trained on massive datasets using self-supervised learning techniques. This allows them to develop a broad understanding of language, images, or other data domains. However, the large scale and general nature of foundation models can also lead to biases, security vulnerabilities, and other issues.
Chapter 3 explains how AI applications can be developed by fine-tuning foundation models for specific tasks. This enables rapid development of capable AI systems, but the risks inherent in the foundation model can then be inherited by the AI application. Data authenticity, consent, and provenance become critical concerns.
Chapter 4 outlines the various risks to trustworthiness that can arise from using foundation models, such as algorithmic bias, privacy violations, and potential misuse. These issues must be carefully evaluated and mitigated when building AI apps on top of foundation models.
The paper then examines the upcoming EU AI Regulation and the requirements it will place on both AI applications and the foundation models they are built upon. Ensuring AI systems respect fundamental rights will be a key focus.
Finally, Chapter 6 outlines a system and procedure for meeting these trustworthiness requirements when developing AI applications with foundation models. This includes risk assessment, testing, and ongoing monitoring and auditing.
Critical Analysis
The whitepaper provides a comprehensive and practical approach for evaluating and ensuring the trustworthiness of AI applications built using foundation models. By adapting the Fraunhofer IAIS framework to this specific context, it offers concrete guidance that can be applied by AI developers and organizations.
However, the paper does not delve deeply into some of the more complex technical and ethical challenges posed by foundation models. For example, it does not explore in detail how to mitigate algorithmic biases that may be present in the underlying foundation model. Nor does it address the potential for foundation models to be misused for harmful purposes, beyond briefly mentioning security vulnerabilities.
Additionally, the paper's focus is on ensuring trustworthiness from the perspective of the AI application developer. It does not consider the concerns and perspectives of end-users, affected communities, or other stakeholders who may be impacted by these AI systems. A more holistic, user-centric approach to trustworthiness could be valuable.
Overall, this whitepaper provides a solid foundation for addressing trustworthiness in foundation model-based AI applications. However, further research and dialogue are needed to fully grapple with the complex ethical and societal implications of this powerful new AI technology.
Conclusion
This whitepaper offers a detailed and practical guide for evaluating and ensuring the trustworthiness of AI applications developed using foundation models. By adapting the Fraunhofer IAIS framework for trustworthy AI to this specific context, it provides a structured approach that AI developers can use to proactively address the risks and challenges posed by foundation models.
The paper covers the key technical and regulatory considerations, outlining the relationship between foundation models and AI apps, the construction of foundation models, the development process for AI apps, the resulting trustworthiness risks, and how to meet the requirements of the upcoming EU AI Regulation.
By taking a risk-based, holistic view of trustworthiness, this framework can help unlock the immense potential of foundation models while ensuring they are deployed responsibly and with appropriate safeguards. As foundation models become increasingly prevalent, this guidance will be crucial for building AI systems that are reliable, transparent, and aligned with human values.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
The Journey to Trustworthy AI- Part 1: Pursuit of Pragmatic Frameworks
Mohamad M Nasr-Azadani, Jean-Luc Chatelain
0
0
This paper reviews Trustworthy Artificial Intelligence (TAI) and its various definitions. Considering the principles respected in any society, TAI is often characterized by a few attributes, some of which have led to confusion in regulatory or engineering contexts. We argue against using terms such as Responsible or Ethical AI as substitutes for TAI. And to help clarify any confusion, we suggest leaving them behind. Given the subjectivity and complexity inherent in TAI, developing a universal framework is deemed infeasible. Instead, we advocate for approaches centered on addressing key attributes and properties such as fairness, bias, risk, security, explainability, and reliability. We examine the ongoing regulatory landscape, with a focus on initiatives in the EU, China, and the USA. We recognize that differences in AI regulations based on geopolitical and geographical reasons pose an additional challenge for multinational companies. We identify risk as a core factor in AI regulation and TAI. For example, as outlined in the EU-AI Act, organizations must gauge the risk level of their AI products to act accordingly (or risk hefty fines). We compare modalities of TAI implementation and how multiple cross-functional teams are engaged in the overall process. Thus, a brute force approach for enacting TAI renders its efficiency and agility, moot. To address this, we introduce our framework Set-Formalize-Measure-Act (SFMA). Our solution highlights the importance of transforming TAI-aware metrics, drivers of TAI, stakeholders, and business/legal requirements into actual benchmarks or tests. Finally, over-regulation driven by panic of powerful AI models can, in fact, harm TAI too. Based on GitHub user-activity data, in 2023, AI open-source projects rose to top projects by contributor account. Enabling innovation in TAI hinges on the independent contributions of the open-source community.
4/9/2024
📊
Data Authenticity, Consent, & Provenance for AI are all broken: what will it take to fix them?
Shayne Longpre, Robert Mahari, Naana Obeng-Marnu, William Brannon, Tobin South, Katy Gero, Sandy Pentland, Jad Kabbara
0
0
New capabilities in foundation models are owed in large part to massive, widely-sourced, and under-documented training data collections. Existing practices in data collection have led to challenges in documenting data transparency, tracing authenticity, verifying consent, privacy, representation, bias, copyright infringement, and the overall development of ethical and trustworthy foundation models. In response, regulation is emphasizing the need for training data transparency to understand foundation models' limitations. Based on a large-scale analysis of the foundation model training data landscape and existing solutions, we identify the missing infrastructure to facilitate responsible foundation model development practices. We examine the current shortcomings of common tools for tracing data authenticity, consent, and documentation, and outline how policymakers, developers, and data creators can facilitate responsible foundation model development by adopting universal data provenance standards.
4/22/2024
🤖
Lessons Learned in Performing a Trustworthy AI and Fundamental Rights Assessment
Marjolein Boonstra, Fr'ed'erick Bruneault, Subrata Chakraborty, Tjitske Faber, Alessio Gallucci, Eleanore Hickman, Gerard Kema, Heejin Kim, Jaap Kooiker, Elisabeth Hildt, Annegret Lamad'e, Emilie Wiinblad Mathez, Florian Moslein, Genien Pathuis, Giovanni Sartor, Marijke Steege, Alice Stocco, Willy Tadema, Jarno Tuimala, Isabel van Vledder, Dennis Vetter, Jana Vetter, Magnus Westerlund, Roberto V. Zicari
0
0
This report shares the experiences, results and lessons learned in conducting a pilot project ``Responsible use of AI'' in cooperation with the Province of Friesland, Rijks ICT Gilde-part of the Ministry of the Interior and Kingdom Relations (BZK) (both in The Netherlands) and a group of members of the Z-Inspection$^{small{circledR}}$ Initiative. The pilot project took place from May 2022 through January 2023. During the pilot, the practical application of a deep learning algorithm from the province of Fr^yslan was assessed. The AI maps heathland grassland by means of satellite images for monitoring nature reserves. Environmental monitoring is one of the crucial activities carried on by society for several purposes ranging from maintaining standards on drinkable water to quantifying the CO2 emissions of a particular state or region. Using satellite imagery and machine learning to support decisions is becoming an important part of environmental monitoring. The main focus of this report is to share the experiences, results and lessons learned from performing both a Trustworthy AI assessment using the Z-Inspection$^{small{circledR}}$ process and the EU framework for Trustworthy AI, and combining it with a Fundamental Rights assessment using the Fundamental Rights and Algorithms Impact Assessment (FRAIA) as recommended by the Dutch government for the use of AI algorithms by the Dutch public authorities.
4/23/2024
🎲
Trust in AI: Progress, Challenges, and Future Directions
Saleh Afroogh, Ali Akbari, Evan Malone, Mohammadali Kargar, Hananeh Alambeigi
0
0
The increasing use of artificial intelligence (AI) systems in our daily life through various applications, services, and products explains the significance of trust/distrust in AI from a user perspective. AI-driven systems (as opposed to other technologies) have ubiquitously diffused in our life not only as some beneficial tools to be used by human agents but also are going to be substitutive agents on our behalf, or manipulative minds that would influence human thought, decision, and agency. Trust/distrust in AI plays the role of a regulator and could significantly control the level of this diffusion, as trust can increase, and distrust may reduce the rate of adoption of AI. Recently, varieties of studies have paid attention to the variant dimension of trust/distrust in AI, and its relevant considerations. In this systematic literature review, after conceptualization of trust in the current AI literature review, we will investigate trust in different types of human-Machine interaction, and its impact on technology acceptance in different domains. In addition to that, we propose a taxonomy of technical (i.e., safety, accuracy, robustness) and non-technical axiological (i.e., ethical, legal, and mixed) trustworthiness metrics, and some trustworthy measurements. Moreover, we examine some major trust-breakers in AI (e.g., autonomy and dignity threat), and trust makers; and propose some future directions and probable solutions for the transition to a trustworthy AI.
4/5/2024