AI Procurement Checklists: Revisiting Implementation in the Age of AI Governance

Overview

• Growing use of AI in the public sector over the past decade
• Efforts to regulate government use of AI have recently gained attention
• Promoting ethical and effective deployment of AI in government is a complex challenge
• Tensions between mitigating risks and making AI adoption easy for resource-constrained public agencies

Plain English Explanation

The use of AI by government agencies has been quietly increasing for years, but it's only recently that there have been serious efforts to regulate this technology in the public sector. While it may seem straightforward to ensure the ethical and effective use of AI systems in government, it's actually a notoriously difficult task.

On one hand, there are significant concerns about the potential downsides of using AI-based tools, such as bias towards marginalized communities, safety issues, and the risk of the systems being gamed. On the other hand, there's pressure not to make it too difficult for government agencies to adopt AI, especially since the public sector often has fewer resources than private companies - the ability to save scarce government resources is often a big reason for using AI tools in the first place.

These competing tensions create a real risk that any regulations or procedures put in place to protect vulnerable groups from being harmed by government use of AI will end up being more performative than truly effective. To inform the latest wave of regulatory efforts in the US, the authors of this paper looked at the lessons learned by officials in other jurisdictions with more mature regulations around government AI use, like Brazil, Singapore, and Canada.

Technical Explanation

The paper investigates two specific checklists that have been implemented to govern AI procurement and use in the public sector: the Canadian Directive on Automated Decision-Making (CDADM) and the World Economic Forum's AI Procurement in a Box (WEF).

Through interviews with government officials, the researchers identified three key pitfalls that can undermine the effectiveness of regulations aimed at ensuring the responsible use of AI by public agencies:

1. Expertise: Ensuring the right experts are involved in the design and implementation of AI governance frameworks is crucial but challenging, as government agencies often lack in-house technical AI expertise.

2. Risk Frameworks: Developing appropriate risk assessment frameworks to categorize different AI use cases and determine the level of scrutiny required is complex, and risks being overly subjective or narrow in scope.

3. Transparency: Striking the right balance between transparency requirements and legitimate concerns about protecting sensitive information or intellectual property is difficult, and can lead to regulations that are more about visible compliance than meaningful oversight.

The paper suggests several avenues for improvement, such as building AI audit standards and boards to provide ongoing guidance, and taking a more systematic approach to mapping the landscape of AI use in government to better inform regulatory efforts.

Critical Analysis

The paper provides a nuanced and well-researched look at the challenges of regulating government use of AI. It rightly highlights the inherent tensions between mitigating risks and making AI adoption feasible for cash-strapped public agencies. The three key pitfalls identified - around expertise, risk frameworks, and transparency - seem like valid and important concerns based on the evidence presented.

One potential limitation of the research is the relatively narrow focus on specific checklist frameworks implemented in a few jurisdictions. While these case studies provide valuable insights, the findings may not be fully generalizable to all regulatory approaches or government contexts.

Additionally, the paper could have delved deeper into potential solutions or best practices to address the identified pitfalls. While it suggests some avenues for improvement, a more comprehensive discussion of possible remedies would have strengthened the overall contribution.

Nevertheless, this paper makes a valuable contribution to the ongoing discourse around responsible AI governance in the public sector. By shedding light on the practical challenges faced by policymakers, it encourages readers to think critically about how to build more durable and effective frameworks for regulating the use of AI in government.

Conclusion

This paper provides a nuanced look at the complexities of regulating government use of AI. It highlights the inherent tensions between mitigating risks and making AI adoption feasible for resource-constrained public agencies, and identifies three key pitfalls - around expertise, risk frameworks, and transparency - that can undermine the effectiveness of regulatory efforts.

While focused on specific case studies, the paper's findings offer important lessons for policymakers and AI governance experts working to ensure the ethical and responsible deployment of AI in the public sector. By shedding light on these practical challenges, the research encourages critical thinking about how to build more durable and effective frameworks for governing the use of AI by government agencies.