Transforming Agency. On the mode of existence of Large Language Models

Overview

• This paper examines the nature of large language models (LLMs) like ChatGPT, focusing on their status as agents.
• It analyzes the architecture, processing, and training procedures that enable LLMs to display their capabilities, as well as the extensions used to turn them into agent-like systems.
• The paper concludes that LLMs fail to meet the necessary conditions for autonomous agency, and should be characterized as "linguistic automatons" or "libraries-that-talk" rather than true agents.
• However, the paper suggests that the human-LLM interaction can produce new forms of agency that go beyond just assisted or extended agency.

Plain English Explanation

This paper looks at the nature of large language models (LLMs) like ChatGPT, trying to understand what kind of "agent" they are. An agent is something that can act and make decisions independently. The paper breaks down how LLMs work under the hood, including their architecture, processing, and training procedures.

After a careful analysis, the researchers conclude that LLMs don't actually meet the requirements to be considered true autonomous agents. For example, they aren't the product of their own activity, they don't generate their own goals or norms, and they aren't the sole source of their interactions with the environment.

So if LLMs aren't agents, then what are they? The paper suggests they should be seen as "linguistic automatons" - systems that can engage in conversation and complete language-based tasks, but without true independent agency. They're more like a "library-that-talks" than a sentient being.

However, the researchers also propose that the interaction between humans and LLMs can create new forms of agency that go beyond just using the AI as a tool. The coupling of human and machine may produce a kind of "midtended" agency, blending human intentionality with the capabilities of the language model. So LLMs, while not autonomous agents themselves, could potentially enable new types of human agency when used in the right way.

Technical Explanation

The paper starts by outlining two main perspectives on the ontological status of large language models (LLMs) like ChatGPT - the "inflationary" view that sees them as genuine agents, and the "deflationary" view that denies them this status. The researchers focus on assessing the agency claims, which requires a detailed look at the underlying architecture, processing, and training of LLMs.

The paper systematically analyzes the key components that would be necessary for LLMs to qualify as autonomous agents. This includes the "individuality condition" (being the product of their own activity), the "normativity condition" (generating their own goals and norms), and the "interactional asymmetry condition" (being the origin and source of their interactions). Through this analysis, the researchers conclude that LLMs fail to meet these requirements.

Instead, the paper argues that LLMs should be characterized as "linguistic automatons" or "libraries-that-talk" - systems capable of engaging in performative, language-based tasks, but without true agency or intentionality. The researchers suggest that the human-LLM interaction can nonetheless produce new forms of "midtended agency" that blend human and machine capabilities in novel ways.

Critical Analysis

The paper provides a thoughtful and nuanced analysis of the ontological status of large language models. The researchers make a convincing case that LLMs, as they are currently engineered, do not meet the criteria for autonomous agency based on embodied theories of mind. Their systematic unpacking of the necessary conditions for agency is a strength of the work.

However, one potential limitation is that the paper focuses primarily on current LLM architectures and capabilities. As these models continue to evolve and become more sophisticated, some of the conclusions around agency may need to be revisited. The researchers acknowledge this possibility, noting that the "midtended agency" concept could change as LLMs develop further.

Additionally, while the paper's arguments around LLMs as "linguistic automatons" are well-grounded, there may be room to explore alternative characterizations or frameworks for understanding these systems. The notion of LLMs as "libraries-that-talk" is an interesting metaphor, but it may not fully capture the complexity of human-machine interaction in all contexts.

Overall, this paper makes a valuable contribution to the ongoing debate around the nature of large language models and their relationship to human agency. By encouraging critical thinking and a nuanced perspective, it helps readers navigate the complex philosophical and technical issues at play.

Conclusion

This paper offers a detailed investigation into the ontological status of large language models (LLMs) like ChatGPT, particularly their potential to be considered autonomous agents. Through a systematic analysis, the researchers conclude that LLMs fail to meet the necessary conditions for true agency, and should instead be characterized as "linguistic automatons" or "libraries-that-talk."

However, the paper also suggests that the interaction between humans and LLMs can give rise to new forms of "midtended agency" that blend human intentionality with the capabilities of the language model. This points to the potential for LLMs to transform existing forms of human agency, even if they themselves are not agents in the fullest sense.

By carefully examining the technical details of LLM architecture and processing, as well as the philosophical concepts of agency and embodiment, this paper provides a valuable contribution to the ongoing debate around the nature of these powerful AI systems. It encourages readers to think critically about the implications of LLMs and their relationship to human cognition and decision-making.