Chain-of-Planned-Behaviour Workflow Elicits Few-Shot Mobility Generation in LLMs

Overview

• Large language models (LLMs) have revolutionized many fields, but their ability to generate human-like behavior has not been extensively explored.
• This gap likely exists because behavioral intentions are influenced by social norms and personal preferences, not just abstract reasoning.
• The researchers developed a workflow called Chain-of-Planned Behavior (CoPB) that leverages the Theory of Planned Behavior to enhance LLMs' ability to reason about mobility intentions.
• CoPB significantly reduces the error rate in mobility intention generation and can be integrated with mechanistic models to improve scalability and reduce costs.
• The researchers also show how CoPB can be used to generate high-quality labels to fine-tune smaller language models, reducing usage costs.

Plain English Explanation

Large language models (LLMs) like GPT-4 have shown impressive capabilities in various tasks, but their ability to generate human-like behavior has not been extensively studied. This is likely because human behavior is influenced by a complex mix of factors, including social norms and personal preferences, not just abstract reasoning.

To address this, the researchers developed a workflow called Chain-of-Planned Behavior (CoPB) that draws on the Theory of Planned Behavior. CoPB helps LLMs better understand the factors that shape human mobility intentions, such as attitudes, social norms, and perceived control.

By incorporating these cognitive structures, CoPB significantly reduced the error rate in predicting people's next movement from 57.8% to just 19.4%. This shows that LLMs can generate much more accurate and human-like mobility behaviors when they consider the full range of factors that influence human decision-making.

To make CoPB more scalable and cost-effective, the researchers explored combining it with mechanistic mobility models, like the gravity model. This hybrid approach reduced the computational cost by 97.7% while maintaining high performance.

The researchers also demonstrated how CoPB can be used to automatically generate high-quality labels for mobility behavior reasoning. These labels can then be used to fine-tune smaller, more cost-effective language models like LLaMA 3-8B, without sacrificing the quality of the generated behaviors.

Technical Explanation

The researchers developed a workflow called Chain-of-Planned Behavior (CoPB) that is inspired by the Theory of Planned Behavior (TPB). TPB suggests that behavioral intentions are influenced by three key factors: attitude, subjective norms, and perceived behavioral control.

CoPB incorporates these cognitive structures to enhance the ability of LLMs to reason about mobility intentions. Specifically, the workflow involves:

1. Modeling the attitude towards different mobility options based on factors like comfort, convenience, and environmental impact.
2. Considering the subjective norms around acceptable mobility behaviors within a given social context.
3. Assessing the perceived behavioral control, which reflects an individual's perceived ease or difficulty of performing a particular mobility behavior.

By explicitly modeling these factors, CoPB substantially reduces the error rate of mobility intention generation from 57.8% to 19.4% compared to a baseline LLM.

To improve the scalability of CoPB, the researchers explored integrating it with mechanistic mobility models, such as the gravity model. This hybrid approach can effectively map the mobility intentions generated by CoPB to physical mobility behaviors, while reducing the computational cost by 97.7%.

Additionally, the researchers showed that the CoPB workflow can be used to automatically generate high-quality labels for mobility behavior reasoning. These labels can then be used to fine-tune smaller language models like LLaMA 3-8B, significantly reducing usage costs without sacrificing the quality of the generated behaviors.

Critical Analysis

The researchers have made a compelling case for the importance of incorporating cognitive structures like attitudes, social norms, and perceived behavioral control to enhance the ability of LLMs to generate human-like mobility behaviors. The substantial reduction in error rate achieved by CoPB is a promising result that highlights the value of this approach.

However, the paper does not extensively explore the potential limitations or caveats of the CoPB workflow. For instance, it would be interesting to understand how well the model generalizes to different cultural contexts or demographic groups, where social norms and personal preferences may vary.

Additionally, while the integration of CoPB with mechanistic models is a clever strategy to improve scalability and reduce costs, the paper does not provide a detailed analysis of the tradeoffs involved. It would be helpful to understand the specific performance characteristics and computational requirements of the hybrid approach compared to standalone CoPB or LLM-based solutions.

Furthermore, the paper focuses primarily on mobility behaviors, but it is unclear how well the CoPB workflow might extend to other domains of human behavior, such as decision-making in the economic or social spheres. Exploring the broader applicability of the approach could strengthen its contribution to the field.

Despite these potential areas for further investigation, the researchers have presented a thoughtful and innovative approach to enhancing the behavioral reasoning capabilities of LLMs. The work has important implications for the development of more human-centric AI systems that can better understand and interact with people in the real world.

Conclusion

The researchers have developed a workflow called Chain-of-Planned Behavior (CoPB) that leverages the cognitive structures of the Theory of Planned Behavior to significantly improve the ability of large language models to generate human-like mobility behaviors. By explicitly modeling factors like attitudes, social norms, and perceived behavioral control, CoPB reduces the error rate in mobility intention generation from 57.8% to just 19.4%.

To improve the scalability and cost-effectiveness of CoPB, the researchers have explored integrating it with mechanistic mobility models, such as the gravity model. This hybrid approach can reduce the computational cost by 97.7% while maintaining high performance.

The researchers have also demonstrated how CoPB can be used to automatically generate high-quality labels for mobility behavior reasoning. These labels can be used to fine-tune smaller, more cost-effective language models like LLaMA 3-8B, without sacrificing the quality of the generated behaviors.

This work has important implications for the development of AI systems that can better understand and interact with people in the real world. By incorporating cognitive structures that shape human decision-making, LLMs can generate more accurate and human-like behaviors, paving the way for more natural and engaging interactions between AI and people.