Addressing the Regulatory Gap: Moving Towards an EU AI Audit Ecosystem Beyond the AIA by Including Civil Society
0
🤖
Sign in to get full access
Overview
- This paper explores the need for a comprehensive EU AI audit ecosystem that goes beyond the proposed AI Act (AIA) by including civil society.
- It argues that the AIA's current auditing framework has regulatory gaps that can be addressed by involving diverse stakeholders in the auditing process.
- The paper proposes a model for a more inclusive and robust AI auditing system to enhance transparency, accountability, and public trust in AI systems.
Plain English Explanation
The paper discusses the importance of having a comprehensive system for auditing AI systems in the European Union (EU). The authors argue that the current proposal for the AI Act (AIA) has some gaps that can be addressed by involving a wider range of stakeholders, including civil society organizations, in the auditing process.
The authors believe that the AIA's auditing framework, which mainly focuses on industry and regulatory bodies, may not be enough to ensure transparency, accountability, and public trust in AI systems. Incorporating diverse perspectives from civil society could help create a more robust and inclusive AI auditing ecosystem.
By expanding access to data and audit processes, the proposed model aims to enhance the public's ability to scrutinize and hold AI systems accountable. This could lead to greater trust in the development and deployment of AI technologies, ultimately benefiting both businesses and consumers.
Technical Explanation
The paper begins by outlining the key properties and terminology associated with algorithm audits, including concepts like transparency, accountability, and the distinction between ex-ante and ex-post audits.
The authors then analyze the current AIA proposal, highlighting how its auditing framework is primarily focused on industry and regulatory bodies. They argue that this approach may not be sufficient to address the broader societal concerns and implications of AI systems.
To address this gap, the paper proposes a model for an EU AI audit ecosystem that includes civil society organizations as active participants. This model emphasizes the importance of data access and collaboration between various stakeholders, such as researchers, policymakers, and impacted communities.
The authors discuss the potential benefits of this expanded auditing ecosystem, including enhanced transparency, increased public trust, and more comprehensive assessments of AI systems' societal impacts.
Critical Analysis
The paper raises valid concerns about the limitations of the AIA's current auditing framework and the need for a more inclusive approach. Involving civil society organizations in the auditing process could indeed lead to more comprehensive and nuanced assessments of AI systems.
However, the paper does not provide detailed implementation details or address potential challenges, such as how to ensure the meaningful participation of civil society groups, manage potential conflicts of interest, or balance the need for transparency with legitimate business confidentiality concerns.
Additionally, the paper could have addressed the potential risks of over-reliance on audits, as they are not a panacea for addressing all AI-related issues. There may be a need for complementary approaches, such as procurement checklists and automated transparency measures, to create a comprehensive regulatory framework.
Conclusion
This paper advocates for a more inclusive and robust AI auditing ecosystem in the EU, which would go beyond the current AIA proposal by actively involving civil society organizations. The authors argue that this expanded approach could enhance transparency, accountability, and public trust in the development and deployment of AI systems.
While the paper raises important considerations, further research and practical implementation details are needed to address the challenges and ensure the effectiveness of this proposed model. Nonetheless, the core idea of creating a more inclusive and comprehensive AI auditing system is a valuable contribution to the ongoing discussions around regulating and governing AI technologies in the EU and beyond.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🤖
0
Addressing the Regulatory Gap: Moving Towards an EU AI Audit Ecosystem Beyond the AIA by Including Civil Society
David Hartmann, Jos'e Renato Laranjeira de Pereira, Chiara Streitborger, Bettina Berendt
The European legislature has proposed the Digital Services Act (DSA) and Artificial Intelligence Act (AIA) to regulate platforms and Artificial Intelligence (AI) products. We review to what extent third-party audits are part of both laws and to what extent access to models and data is provided. By considering the value of third-party audits and third-party data access in an audit ecosystem, we identify a regulatory gap in that the Artificial Intelligence Act does not provide access to data for researchers and civil society. Our contributions to the literature include: (1) Defining an AI audit ecosystem that incorporates compliance and oversight. (2) Highlighting a regulatory gap within the DSA and AIA regulatory framework, preventing the establishment of an AI audit ecosystem. (3) Emphasizing that third-party audits by research and civil society must be part of that ecosystem and demand that the AIA include data and model access for certain AI products. We call for the DSA to provide NGOs and investigative journalists with data access to platforms by delegated acts and for adaptions and amendments of the AIA to provide third-party audits and data and model access at least for high-risk systems to close the regulatory gap. Regulations modeled after European Union AI regulations should enable data access and third-party audits, fostering an AI audit ecosystem that promotes compliance and oversight mechanisms.
Read more5/21/2024
🤔
0
Law and the Emerging Political Economy of Algorithmic Audits
Petros Terzis, Michael Veale, Noelle Gaumann
For almost a decade now, scholarship in and beyond the ACM FAccT community has been focusing on novel and innovative ways and methodologies to audit the functioning of algorithmic systems. Over the years, this research idea and technical project has matured enough to become a regulatory mandate. Today, the Digital Services Act (DSA) and the Online Safety Act (OSA) have established the framework within which technology corporations and (traditional) auditors will develop the `practice' of algorithmic auditing thereby presaging how this `ecosystem' will develop. In this paper, we systematically review the auditing provisions in the DSA and the OSA in light of observations from the emerging industry of algorithmic auditing. Who is likely to occupy this space? What are some political and ethical tensions that are likely to arise? How are the mandates of `independent auditing' or `the evaluation of the societal context of an algorithmic function' likely to play out in practice? By shaping the picture of the emerging political economy of algorithmic auditing, we draw attention to strategies and cultures of traditional auditors that risk eroding important regulatory pillars of the DSA and the OSA. Importantly, we warn that ambitious research ideas and technical projects of/for algorithmic auditing may end up crashed by the standardising grip of traditional auditors and/or diluted within a complex web of (sub-)contractual arrangements, diverse portfolios, and tight timelines.
Read more6/19/2024
🏷️
0
A Framework for Assurance Audits of Algorithmic Systems
Khoa Lam, Benjamin Lange, Borhane Blili-Hamelin, Jovana Davidovic, Shea Brown, Ali Hasan
An increasing number of regulations propose AI audits as a mechanism for achieving transparency and accountability for artificial intelligence (AI) systems. Despite some converging norms around various forms of AI auditing, auditing for the purpose of compliance and assurance currently lacks agreed-upon practices, procedures, taxonomies, and standards. We propose the criterion audit as an operationalizable compliance and assurance external audit framework. We model elements of this approach after financial auditing practices, and argue that AI audits should similarly provide assurance to their stakeholders about AI organizations' ability to govern their algorithms in ways that mitigate harms and uphold human values. We discuss the necessary conditions for the criterion audit and provide a procedural blueprint for performing an audit engagement in practice. We illustrate how this framework can be adapted to current regulations by deriving the criteria on which bias audits can be performed for in-scope hiring algorithms, as required by the recently effective New York City Local Law 144 of 2021. We conclude by offering a critical discussion on the benefits, inherent limitations, and implementation challenges of applying practices of the more mature financial auditing industry to AI auditing where robust guardrails against quality assurance issues are only starting to emerge. Our discussion -- informed by experiences in performing these audits in practice -- highlights the critical role that an audit ecosystem plays in ensuring the effectiveness of audits.
Read more5/29/2024
0
The Necessity of AI Audit Standards Boards
David Manheim, Sammy Martin, Mark Bailey, Mikhail Samin, Ross Greutzmacher
Auditing of AI systems is a promising way to understand and manage ethical problems and societal risks associated with contemporary AI systems, as well as some anticipated future risks. Efforts to develop standards for auditing Artificial Intelligence (AI) systems have therefore understandably gained momentum. However, we argue that creating auditing standards is not just insufficient, but actively harmful by proliferating unheeded and inconsistent standards, especially in light of the rapid evolution and ethical and safety challenges of AI. Instead, the paper proposes the establishment of an AI Audit Standards Board, responsible for developing and updating auditing methods and standards in line with the evolving nature of AI technologies. Such a body would ensure that auditing practices remain relevant, robust, and responsive to the rapid advancements in AI. The paper argues that such a governance structure would also be helpful for maintaining public trust in AI and for promoting a culture of safety and ethical responsibility within the AI industry. Throughout the paper, we draw parallels with other industries, including safety-critical industries like aviation and nuclear energy, as well as more prosaic ones such as financial accounting and pharmaceuticals. AI auditing should emulate those fields, and extend beyond technical assessments to include ethical considerations and stakeholder engagement, but we explain that this is not enough; emulating other fields' governance mechanisms for these processes, and for audit standards creation, is a necessity. We also emphasize the importance of auditing the entire development process of AI systems, not just the final products...
Read more4/23/2024