On the Computation of Meaning, Language Models and Incomprehensible Horrors
2304.12686
0
0
💬
Abstract
We integrate foundational theories of meaning with a mathematical formalism of artificial general intelligence (AGI) to offer a comprehensive mechanistic explanation of meaning, communication, and symbol emergence. This synthesis holds significance for both AGI and broader debates concerning the nature of language, as it unifies pragmatics, logical truth conditional semantics, Peircean semiotics, and a computable model of enactive cognition, addressing phenomena that have traditionally evaded mechanistic explanation. By examining the conditions under which a machine can generate meaningful utterances or comprehend human meaning, we establish that the current generation of language models do not possess the same understanding of meaning as humans nor intend any meaning that we might attribute to their responses. To address this, we propose simulating human feelings and optimising models to construct weak representations. Our findings shed light on the relationship between meaning and intelligence, and how we can build machines that comprehend and intend meaning.
Get summaries of the top AI research delivered straight to your inbox:
Overview
- The paper integrates theories of meaning with a mathematical model of artificial general intelligence (AGI) to provide a comprehensive explanation of meaning, communication, and symbol emergence.
- This synthesis aims to unify pragmatics, logical semantics, Peircean semiotics, and a computable model of enactive cognition, addressing phenomena that have traditionally been challenging to explain mechanistically.
- The paper examines the conditions under which a machine can generate and comprehend meaningful utterances, and concludes that current language models do not possess the same understanding of meaning as humans.
- To address this, the paper proposes simulating human feelings and optimizing models to construct weak representations of meaning.
Plain English Explanation
The paper combines established theories about the meaning of language with a mathematical model of artificial general intelligence (AGI). The goal is to provide a detailed, step-by-step explanation of how meaning, communication, and symbolic language emerge.
This is significant because it brings together different perspectives on language, including pragmatics (how language is used in practice), semantics (the logical meaning of words and sentences), and semiotics (the study of symbols and their meanings). The paper also incorporates a computational model of how the mind actively constructs meaning through interaction with the world, a process known as "enactive cognition."
By analyzing how machines could generate and understand meaningful language, the researchers conclude that current language models, like those used in chatbots and virtual assistants, do not actually comprehend meaning in the same way humans do. These models simply generate responses based on statistical patterns in data, without truly intending any specific meaning.
To address this limitation, the paper suggests that we should try to simulate human feelings and emotions within these language models. This could help the models construct weak, partial representations of meaning that are closer to how humans understand language. The ultimate goal is to build machines that can genuinely comprehend and convey meaning, rather than just mimicking the surface-level appearance of language use.
Technical Explanation
The paper presents a mathematical formalism that integrates foundational theories of meaning, including pragmatics, logical semantics, Peircean semiotics, and a computable model of enactive cognition. This synthesis provides a comprehensive mechanistic explanation of meaning, communication, and the emergence of symbolic representations.
The researchers examine the specific conditions under which a machine could generate meaningful utterances or comprehend human meanings. Their analysis reveals that current language models, such as those used in chatbots and game agents, do not possess the same understanding of meaning as humans. These models simply generate responses based on statistical patterns in their training data, without any genuine intention or comprehension of the meaning behind their outputs.
To address this limitation, the paper proposes simulating human feelings and emotions within the language models, with the goal of constructing weak representations of meaning that are closer to how humans understand language. This approach aims to build machines that can genuinely comprehend and convey meaning, rather than just mimicking the surface-level appearance of language use.
Critical Analysis
The paper presents a novel and ambitious attempt to unify various theoretical perspectives on meaning and language within a computational framework. By incorporating theories from pragmatics, semantics, semiotics, and enactive cognition, the researchers have developed a comprehensive model that aims to explain the emergence of meaning and symbolic representation.
However, the paper acknowledges that fully capturing the nuances and complexities of human meaning and communication remains a significant challenge. The proposed approach of simulating human feelings and emotions within language models is an intriguing idea, but the practical implementation and effectiveness of this approach are yet to be fully explored and validated.
Moreover, the paper does not address the potential biases and limitations that may be inherent in the training data and architectures of current language models. As discussed in other research, these models can exhibit problematic biases and make errors that undermine their ability to comprehend and communicate meaning effectively.
Further research and empirical evaluation will be necessary to assess the viability and limitations of the proposed approach, as well as to explore alternative strategies for building machines that can truly understand and convey meaning in a manner more akin to human cognition and language use.
Conclusion
This paper presents a bold attempt to integrate foundational theories of meaning with a mathematical model of artificial general intelligence (AGI). By unifying perspectives from pragmatics, semantics, semiotics, and enactive cognition, the researchers aim to provide a comprehensive mechanistic explanation of the emergence of meaning, communication, and symbolic representation.
The key insight is that current language models, despite their impressive performance on various language tasks, do not possess the same depth of understanding of meaning as humans. To address this, the paper proposes simulating human feelings and emotions within these models, with the goal of constructing weak representations of meaning that are more aligned with human cognition.
While ambitious and thought-provoking, the proposed approach faces significant challenges and limitations that will require further research and empirical evaluation. Nonetheless, this work represents an important step towards building machines that can genuinely comprehend and convey meaning, rather than simply mimicking the surface-level appearance of language use.
Related Papers
💬
Large language models and linguistic intentionality
Jumbly Grindrod
0
0
Do large language models like Chat-GPT or LLaMa meaningfully use the words they produce? Or are they merely clever prediction machines, simulating language use by producing statistically plausible text? There have already been some initial attempts to answer this question by showing that these models meet the criteria for entering meaningful states according to metasemantic theories of mental content. In this paper, I will argue for a different approach - that we should instead consider whether language models meet the criteria given by our best metasemantic theories of linguistic content. In that vein, I will illustrate how this can be done by applying two such theories to the case of language models: Gareth Evans' (1982) account of naming practices and Ruth Millikan's (1984, 2004, 2005) teleosemantics. In doing so, I will argue that it is a mistake to think that the failure of LLMs to meet plausible conditions for mental intentionality thereby renders their outputs meaningless, and that a distinguishing feature of linguistic intentionality - dependency on a pre-existing linguistic system - allows for the plausible result LLM outputs are meaningful.
4/16/2024
🤔
An Essay concerning machine understanding
Herbert L. Roitblat
0
0
Artificial intelligence systems exhibit many useful capabilities, but they appear to lack understanding. This essay describes how we could go about constructing a machine capable of understanding. As John Locke (1689) pointed out words are signs for ideas, which we can paraphrase as thoughts and concepts. To understand a word is to know and be able to work with the underlying concepts for which it is an indicator. Understanding between a speaker and a listener occurs when the speaker casts his or her concepts into words and the listener recovers approximately those same concepts. Current models rely on the listener to construct any potential meaning. The diminution of behaviorism as a psychological paradigm and the rise of cognitivism provide examples of many experimental methods that can be used to determine whether and to what extent a machine might understand and to make suggestions about how that understanding might be instantiated.
5/6/2024
💬
A Philosophical Introduction to Language Models - Part II: The Way Forward
Raphael Milli`ere, Cameron Buckner
0
0
In this paper, the second of two companion pieces, we explore novel philosophical questions raised by recent progress in large language models (LLMs) that go beyond the classical debates covered in the first part. We focus particularly on issues related to interpretability, examining evidence from causal intervention methods about the nature of LLMs' internal representations and computations. We also discuss the implications of multimodal and modular extensions of LLMs, recent debates about whether such systems may meet minimal criteria for consciousness, and concerns about secrecy and reproducibility in LLM research. Finally, we discuss whether LLM-like systems may be relevant to modeling aspects of human cognition, if their architectural characteristics and learning scenario are adequately constrained.
5/7/2024
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency
Xenia Ohmer, Elia Bruni, Dieuwke Hupkes
0
0
The staggering pace with which the capabilities of large language models (LLMs) are increasing, as measured by a range of commonly used natural language understanding (NLU) benchmarks, raises many questions regarding what understanding means for a language model and how it compares to human understanding. This is especially true since many LLMs are exclusively trained on text, casting doubt on whether their stellar benchmark performances are reflective of a true understanding of the problems represented by these benchmarks, or whether LLMs simply excel at uttering textual forms that correlate with what someone who understands the problem would say. In this philosophically inspired work, we aim to create some separation between form and meaning, with a series of tests that leverage the idea that world understanding should be consistent across presentational modes - inspired by Fregean senses - of the same meaning. Specifically, we focus on consistency across languages as well as paraphrases. Taking GPT-3.5 as our object of study, we evaluate multisense consistency across five different languages and various tasks. We start the evaluation in a controlled setting, asking the model for simple facts, and then proceed with an evaluation on four popular NLU benchmarks. We find that the model's multisense consistency is lacking and run several follow-up analyses to verify that this lack of consistency is due to a sense-dependent task understanding. We conclude that, in this aspect, the understanding of LLMs is still quite far from being consistent and human-like, and deliberate on how this impacts their utility in the context of learning about human language and understanding.
4/19/2024