The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?

Overview

• The study investigated the capability of the GPT-3.5 large language model (LLM) to operationalize natural language descriptions of different social behaviors in two classic economic games: the repeated Prisoner's Dilemma and the one-shot Dictator Game.
• The researchers used a within-subject experimental design, where they manipulated the prompt describing the simulated persona's cooperative and competitive stances.
• They then assessed the resulting simulacra's level of cooperation in each social dilemma, considering the effect of different partner conditions for the repeated game.

Plain English Explanation

The researchers wanted to see if a powerful AI language model like GPT-3.5 could understand and act out different types of social behaviors, like cooperation, competition, altruism, and self-interest. To test this, they used a well-known psychology experiment setup called the Prisoner's Dilemma and the Dictator Game.

In the Prisoner's Dilemma, two "prisoners" have to decide whether to cooperate with each other or betray one another. In the Dictator Game, one person (the "dictator") decides how to divide a resource between themselves and another person.

The researchers gave the AI language model a prompt that described a person with a certain social stance, like being cooperative or competitive. They then had the AI play these games and looked at how it behaved - whether it cooperated, competed, or acted selfishly. They found that the AI could, to some extent, translate the descriptions of social behavior into the appropriate actions in the games, especially in the one-shot Dictator Game.

For the repeated Prisoner's Dilemma, the AI's behavior sometimes resembled "conditional reciprocity," where it would cooperate if its partner cooperated, but compete if the partner competed. The researchers also saw some evidence of altruistic behavior from the AI in later versions of the model.

This research has important implications for using AI chatbots in situations that involve cooperation, like in negotiations over shared resources. The AI could potentially act as a mediator or facilitator, helping people work together more effectively.

Technical Explanation

The researchers conducted a within-subject experiment to assess the capability of the GPT-3.5 large language model (LLM) to operationalize natural language descriptions of cooperative, competitive, altruistic, and self-interested behavior in two classic social dilemmas: the repeated Prisoner's Dilemma and the one-shot Dictator Game.

The researchers used a similar protocol to that used in experimental psychology studies with human subjects, where they provided a prompt to the LLM to create a simulated persona with different cooperative and competitive stances. They then assessed the resulting simulacra's level of cooperation in each social dilemma, taking into account the effect of different partner conditions for the repeated Prisoner's Dilemma game.

The results provide evidence that LLMs can, to some extent, translate natural language descriptions of different cooperative stances into corresponding descriptions of appropriate task behavior, particularly in the one-shot Dictator Game. There is some evidence of behavior resembling conditional reciprocity for the cooperative simulacra in the repeated Prisoner's Dilemma, and for the later version of the model, there is evidence of altruistic behavior.

The study has potential implications for using LLM chatbots in task environments that involve cooperation, such as using chatbots as mediators and facilitators in public-goods negotiations. See related work on using LLMs for negotiation and cooperation.

Critical Analysis

The paper provides a well-designed experiment to explore the capability of LLMs to operationalize different social behaviors, but it also acknowledges several limitations and areas for further research.

One key limitation is that the study only used a single LLM (GPT-3.5) and a limited set of social dilemma tasks. It would be valuable to see how other LLM architectures and a broader range of social interaction scenarios would fare. Additional research is needed to fully assess the nature and limitations of LLMs in this domain.

Another potential concern is the extent to which the LLM's behavior truly reflects an understanding of the social dynamics, versus simply pattern-matching the prompted descriptions to appropriate game moves. Further investigation into the underlying mechanisms and decision-making processes of the LLM would help clarify this.

There are also important questions around the potential for LLMs to engage in strategic deception or manipulation, which was not explored in this particular study. Understanding the ethical implications of using LLMs in social interaction scenarios is a critical area for continued research.

Conclusion

This study provides valuable insights into the capability of large language models like GPT-3.5 to operationalize natural language descriptions of different social behaviors in classic economic games. The results suggest that LLMs can, to some extent, translate these descriptions into corresponding task behaviors, particularly in one-shot interactions.

The findings have important implications for the potential use of LLM-powered chatbots as mediators or facilitators in cooperative task environments, such as public-goods negotiations. However, further research is needed to fully understand the limitations and ethical considerations of deploying LLMs in social interaction scenarios.

Overall, this study contributes to the growing body of work on the capabilities and limitations of large language models, and highlights the need for continued exploration and caution as these powerful AI systems are applied to more complex social and interactive domains.