Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use

Overview

• The paper examines how decisions to adopt AI tools in public sector agencies are made in practice
• Researchers interviewed 16 agency decision-makers to understand how their interactions and assumptions about other stakeholders influence AI design and adoption
• The findings shed light on the barriers and disincentives to involving a broader range of stakeholders in these decisions

Plain English Explanation

As more and more public sector agencies start using AI tools for important tasks like social services, it's crucial to understand how they decide to adopt these tools. The researchers in this study [<a href="https://aimodels.fyi/papers/arxiv/automatic-authorities-power-ai" target="_blank">1</a>] looked at this problem from a unique angle - they "studied up" and focused on the decision-makers in these agencies, rather than the typical approach of studying the end users or impacted communities.

Through interviews and activities with 16 agency decision-makers, the researchers examined how these leaders' interactions and assumptions about other stakeholders, such as frontline workers, agency leaders, legal systems, and contracted companies, influence their decisions around AI design and adoption. They found that factors like legal constraints, power dynamics, and knowledge gaps create barriers to involving a wider range of people in these important decisions.

The decision-makers expressed a desire for more practical support to help them better engage with diverse stakeholders, as they recognized the challenges in bridging the divides between them and groups like frontline workers or community members. By centering these power dynamics, the study [<a href="https://aimodels.fyi/papers/arxiv/ai-procurement-checklists-revisiting-implementation-age-ai" target="_blank">2</a>] provides insights that can inform future research and policy efforts to make public sector AI adoption more participatory and inclusive.

Technical Explanation

The researchers conducted semi-structured interviews and design activities with 16 decision-makers from public sector agencies to understand how they make choices about adopting AI tools. They focused on exploring the networks of power relations that shape these decisions, examining the interactions and assumptions that agency leaders have about other stakeholders, such as frontline workers, agency leadership, legal systems, and contracted companies.

Through their analysis, the researchers identified several key factors that create barriers and disincentives to involving a broader range of stakeholders in decisions about AI design and adoption. These include legal and regulatory constraints, knowledge and power differentials between decision-makers and other groups like frontline workers or impacted communities, and the complex web of relationships and dependencies within these agencies.

The decision-makers expressed a desire for more practical support and resources to help them navigate these challenges and better engage with diverse stakeholders [<a href="https://aimodels.fyi/papers/arxiv/bridging-gap-towards-expanded-toolkit-ml-supported" target="_blank">3</a>]. By centering the power dynamics at play, the study [<a href="https://aimodels.fyi/papers/arxiv/deconstructing-human-ai-collaboration-agency-interaction-adaptation" target="_blank">4</a>] provides a nuanced understanding of the barriers to participatory approaches in public sector AI, with implications for future research and policy efforts in this area.

Critical Analysis

The paper offers valuable insights into the decision-making processes around public sector AI adoption, shedding light on the power dynamics and systemic barriers that often get overlooked. By focusing on the agency decision-makers, the researchers were able to uncover a unique perspective on the challenges of stakeholder engagement and the need for more practical support in this area.

However, the study is limited in scope, as it only involved 16 participants from a relatively small sample of public sector agencies. While the findings provide a rich and detailed account of the decision-making process, the generalizability of the results may be limited. Additionally, the paper does not delve deeply into the specific legal, regulatory, or organizational factors that shape these decisions, which could be an area for further exploration [<a href="https://aimodels.fyi/papers/arxiv/power-play-investigating-license-to-critique-teams" target="_blank">5</a>].

Overall, the study serves as an important contribution to the growing body of research on public sector AI, highlighting the need for a more nuanced understanding of the power dynamics and structural barriers that can hinder the adoption of more participatory and inclusive approaches. By continuing to critically examine these issues, researchers and policymakers can work towards developing more equitable and accountable AI systems in the public sector.

Conclusion

This paper offers a unique perspective on the decision-making processes around public sector AI adoption, focusing on the agency decision-makers and the power dynamics that shape their interactions with other stakeholders. The findings shed light on the barriers and disincentives to involving a broader range of people in these important decisions, highlighting the need for more practical support and resources to help bridge the knowledge and power divides.

By centering the networks of power relations, the study provides valuable insights that can inform future research and policy efforts to make public sector AI adoption more participatory and inclusive. As AI tools continue to be rapidly introduced in high-stakes domains, understanding the complex factors that influence these decisions will be crucial for ensuring that these technologies are designed and implemented in a way that prioritizes the needs and concerns of the communities they are meant to serve.