Learning mental states estimation through self-observation: a developmental synergy between intentions and beliefs representations in a deep-learning model of Theory of Mind

Read original: arXiv:2407.18022 - Published 7/26/2024 by Francesca Bianco, Silvia Rigato, Maria Laura Filippetti, Dimitri Ognibene

📈

Overview

This paper explores the relationship between learning to predict others' low-level mental states (like intentions and goals) and attributing higher-level mental states (like beliefs).
The researchers use a simple deep learning model to show that learning to attribute beliefs can occur by observing one's own decision-making processes in partially observable environments.
They find that more accurate predictions of others' intentions and actions can be made earlier when beliefs attribution is learned simultaneously.
The learning performance also improves when observing actors with different embodiments, and the gain is higher when observing beliefs-driven behaviors.
The researchers propose this computational approach can inform our understanding of human social cognitive development and be relevant for designing adaptive social robots.

Plain English Explanation

The paper explores how humans develop the ability to understand others' mental states, which is crucial for effective social interaction. Specifically, it looks at the relationship between learning to predict others' intentions and goals (low-level mental states) and attributing higher-level mental states like beliefs.

The researchers use a simple deep learning model to show that learning to attribute beliefs can happen by observing one's own decision-making processes in situations where not all information is available. They find that when the model learns to predict others' intentions and actions, it can make more accurate predictions earlier if it also learns to attribute beliefs at the same time.

Furthermore, the model's learning performance improves even when observing actors with different physical embodiments, and the gain is even greater when observing beliefs-driven behaviors.

The researchers suggest this computational approach can help us better understand how humans develop the ability to understand others' minds and can be useful for designing social robots that can autonomously understand, assist, and learn from human partners in natural environments and tasks.

Technical Explanation

The paper presents a computational model that explores the relationship between learning to predict low-level mental states (like intentions and goals) and attributing high-level mental states (like beliefs) in the context of Theory of Mind (ToM).

The researchers use a simple feed-forward deep learning model to show that learning to attribute beliefs can occur by observing one's own decision-making processes in partially observable environments. The model is trained to predict the intentions and actions of others based on their observed behavior.

The key finding is that more accurate predictions of others' intentions and actions can be acquired earlier if the model learns to attribute beliefs simultaneously. Additionally, the learning performance improves even when the observed actors have a different embodiment than the observer, and the gain is higher when observing beliefs-driven chunks of behavior.

The researchers propose that this computational approach can inform our understanding of human social cognitive development and be relevant for the design of future adaptive social robots that can autonomously understand, assist, and learn from human interaction partners in novel natural environments and tasks.

Critical Analysis

The paper presents a compelling computational model that sheds light on the relationship between learning low-level and high-level mental state attribution. The researchers acknowledge that their model is relatively simple, and they suggest that more complex architectures and training paradigms may be necessary to fully capture the nuances of human Theory of Mind development.

One potential limitation of the study is that it focuses on a specific type of environment (partially observable) and a specific type of mental state attribution (beliefs). It would be interesting to see how the model performs in a wider range of scenarios and with different types of mental state representations.

Additionally, the researchers do not explicitly address the issue of embodiment and how it might affect the transfer of learning between different agents. While they show that the model can benefit from observing actors with different embodiments, the underlying mechanisms of this process are not fully explored.

Overall, the paper makes a valuable contribution to the understanding of Theory of Mind development and its implications for the design of social robots. The computational approach presented here could serve as a foundation for future research in this important area.

Conclusion

This paper explores the relationship between learning to predict low-level mental states, such as intentions and goals, and attributing higher-level mental states, such as beliefs, in the context of Theory of Mind. The researchers use a simple deep learning model to demonstrate that learning to attribute beliefs can occur by observing one's own decision-making processes in partially observable environments.

The key findings show that more accurate predictions of others' intentions and actions can be made earlier when beliefs attribution is learned simultaneously. Furthermore, the learning performance improves even when observing actors with different embodiments, and the gain is higher when observing beliefs-driven behaviors.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

📈

Learning mental states estimation through self-observation: a developmental synergy between intentions and beliefs representations in a deep-learning model of Theory of Mind

Francesca Bianco, Silvia Rigato, Maria Laura Filippetti, Dimitri Ognibene

Theory of Mind (ToM), the ability to attribute beliefs, intentions, or mental states to others, is a crucial feature of human social interaction. In complex environments, where the human sensory system reaches its limits, behaviour is strongly driven by our beliefs about the state of the world around us. Accessing others' mental states, e.g., beliefs and intentions, allows for more effective social interactions in natural contexts. Yet, these variables are not directly observable, making understanding ToM a challenging quest of interest for different fields, including psychology, machine learning and robotics. In this paper, we contribute to this topic by showing a developmental synergy between learning to predict low-level mental states (e.g., intentions, goals) and attributing high-level ones (i.e., beliefs). Specifically, we assume that learning beliefs attribution can occur by observing one's own decision processes involving beliefs, e.g., in a partially observable environment. Using a simple feed-forward deep learning model, we show that, when learning to predict others' intentions and actions, more accurate predictions can be acquired earlier if beliefs attribution is learnt simultaneously. Furthermore, we show that the learning performance improves even when observed actors have a different embodiment than the observer and the gain is higher when observing beliefs-driven chunks of behaviour. We propose that our computational approach can inform the understanding of human social cognitive development and be relevant for the design of future adaptive social robots able to autonomously understand, assist, and learn from human interaction partners in novel natural environments and tasks.

7/26/2024

Language Models Represent Beliefs of Self and Others

Wentao Zhu, Zhining Zhang, Yizhou Wang

Understanding and attributing mental states, known as Theory of Mind (ToM), emerges as a fundamental capability for human social reasoning. While Large Language Models (LLMs) appear to possess certain ToM abilities, the mechanisms underlying these capabilities remain elusive. In this study, we discover that it is possible to linearly decode the belief status from the perspectives of various agents through neural activations of language models, indicating the existence of internal representations of self and others' beliefs. By manipulating these representations, we observe dramatic changes in the models' ToM performance, underscoring their pivotal role in the social reasoning process. Additionally, our findings extend to diverse social reasoning tasks that involve different causal inference patterns, suggesting the potential generalizability of these representations.

5/31/2024

Explicit Modelling of Theory of Mind for Belief Prediction in Nonverbal Social Interactions

Matteo Bortoletto, Constantin Ruhdorfer, Lei Shi, Andreas Bulling

We propose MToMnet - a Theory of Mind (ToM) neural network for predicting beliefs and their dynamics during human social interactions from multimodal input. ToM is key for effective nonverbal human communication and collaboration, yet, existing methods for belief modelling have not included explicit ToM modelling or have typically been limited to one or two modalities. MToMnet encodes contextual cues (scene videos and object locations) and integrates them with person-specific cues (human gaze and body language) in a separate MindNet for each person. Inspired by prior research on social cognition and computational ToM, we propose three different MToMnet variants: two involving fusion of latent representations and one involving re-ranking of classification scores. We evaluate our approach on two challenging real-world datasets, one focusing on belief prediction, while the other examining belief dynamics prediction. Our results demonstrate that MToMnet surpasses existing methods by a large margin while at the same time requiring a significantly smaller number of parameters. Taken together, our method opens up a highly promising direction for future work on artificial intelligent systems that can robustly predict human beliefs from their non-verbal behaviour and, as such, more effectively collaborate with humans.

8/29/2024

Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models

Chani Jung, Dongkwan Kim, Jiho Jin, Jiseon Kim, Yeon Seonwoo, Yejin Choi, Alice Oh, Hyunwoo Kim

While humans naturally develop theory of mind (ToM), the capability to understand other people's mental states and beliefs, state-of-the-art large language models (LLMs) underperform on simple ToM benchmarks. We posit that we can extend our understanding of LLMs' ToM abilities by evaluating key human ToM precursors -- perception inference and perception-to-belief inference -- in LLMs. We introduce two datasets, Percept-ToMi and Percept-FANToM, to evaluate these precursory inferences for ToM in LLMs by annotating characters' perceptions on ToMi and FANToM, respectively. Our evaluation of eight state-of-the-art LLMs reveals that the models generally perform well in perception inference while exhibiting limited capability in perception-to-belief inference (e.g., lack of inhibitory control). Based on these results, we present PercepToM, a novel ToM method leveraging LLMs' strong perception inference capability while supplementing their limited perception-to-belief inference. Experimental results demonstrate that PercepToM significantly enhances LLM's performance, especially in false belief scenarios.

7/10/2024