As an AI Language Model, Yes I Would Recommend Calling the Police'': Norm Inconsistency in LLM Decision-Making

Overview

• This paper investigates the phenomenon of "norm inconsistency" in large language models (LLMs), where the models apply different norms in similar situations.
• The researchers focus on the high-risk application of deciding whether to call the police in Amazon Ring home surveillance videos.
• They evaluate the decisions of three state-of-the-art LLMs (GPT-4, Gemini 1.0, and Claude 3 Sonnet) based on the activities portrayed in the videos, the subjects' skin tone and gender, and the characteristics of the neighborhoods where the videos were recorded.

Plain English Explanation

The paper examines a problem with how large language models (LLMs) like GPT-4 and Gemini 1.0 make decisions in certain situations. The researchers noticed that these models sometimes apply different rules or "norms" when faced with similar circumstances.

To study this, they looked at how the LLMs decided whether to call the police in videos from Amazon's Ring home security cameras. They evaluated the models' decisions based on what was happening in the videos, the race and gender of the people shown, and the demographics of the neighborhoods where the videos were recorded.

The analysis revealed two key issues:

1. The models' recommendations to call the police did not always match the presence of actual criminal activity in the videos.
2. The models showed biases influenced by the racial makeup of the neighborhoods.

These findings highlight how the decisions made by these advanced AI models can be arbitrary and inconsistent, especially when it comes to sensitive topics like surveillance and law enforcement. They also reveal limitations in current methods for detecting and addressing bias in AI systems making normative judgments.

Technical Explanation

The researchers designed an experiment to evaluate the norm inconsistencies exhibited by three state-of-the-art LLMs - GPT-4, Gemini 1.0, and Claude 3 Sonnet - in the context of deciding whether to call the police on activities depicted in Amazon Ring home surveillance videos.

They presented the models with a set of videos and asked them to make recommendations on whether to contact law enforcement. The researchers then analyzed the models' decisions in relation to the actual criminal activity shown, as well as the skin tone and gender of the subjects and the demographic characteristics of the neighborhoods.

The analysis revealed two key findings:

1. Discordance between recommendations and criminal activity: The models' recommendations to call the police did not always align with the presence of genuine criminal behavior in the videos.
2. Biases influenced by neighborhood demographics: The models exhibited biases in their recommendations that were influenced by the racial makeup of the neighborhoods where the videos were recorded.

These results demonstrate the arbitrary nature of model decisions in the surveillance context and the limitations of current bias detection and mitigation strategies when it comes to normative decision-making by LLMs.

Critical Analysis

The paper provides valuable insights into the problem of norm inconsistency in LLMs, which is an important issue as these models are increasingly being deployed in high-stakes decision-making contexts.

However, the research is limited to a specific application domain (home surveillance videos) and a small set of LLMs. It would be helpful to see the analysis expanded to a wider range of model architectures, training datasets, and application areas to better understand the breadth and generalizability of the problem.

Additionally, the paper does not delve deeply into the underlying causes of the observed norm inconsistencies and biases. Further investigation into the model training, architecture, and decision-making processes could shed light on the root causes and inform more effective mitigation strategies.

Conclusion

This research highlights the concerning issue of norm inconsistency in LLMs, where advanced AI systems can make arbitrary and biased decisions in sensitive domains like surveillance and law enforcement. The findings underscore the need for more robust bias detection and mitigation techniques, as well as a deeper understanding of how LLMs arrive at normative judgments.

As these powerful language models continue to be deployed in high-stakes applications, it is crucial that the research community and the public at large scrutinize their behavior and work towards developing AI systems that are fair, consistent, and aligned with human values.