Explainable Human-AI Interaction: A Planning Perspective
2405.15804
0
0
Abstract
From its inception, AI has had a rather ambivalent relationship with humans -- swinging between their augmentation and replacement. Now, as AI technologies enter our everyday lives at an ever increasing pace, there is a greater need for AI systems to work synergistically with humans. One critical requirement for such synergistic human-AI interaction is that the AI systems be explainable to the humans in the loop. To do this effectively, AI agents need to go beyond planning with their own models of the world, and take into account the mental model of the human in the loop. Drawing from several years of research in our lab, we will discuss how the AI agent can use these mental models to either conform to human expectations, or change those expectations through explanatory communication. While the main focus of the book is on cooperative scenarios, we will point out how the same mental models can be used for obfuscation and deception. Although the book is primarily driven by our own research in these areas, in every chapter, we will provide ample connections to relevant research from other groups.
Create account to get full access
Overview
- This paper provides a literature review on the transition from explainable AI (XAI) to interactive AI systems.
- It examines how AI systems can be designed to improve human-AI collaboration, trust, and understanding.
- The review covers topics such as explanation generation, human-autonomy teaming, and incorporating explanations into human-machine interfaces.
Plain English Explanation
The paper discusses how AI systems can be designed to be more transparent and interactive, helping humans better understand and work alongside them. Rather than just providing opaque "black box" outputs, the review examines ways for AI to offer explanations for its decisions and behavior. This could involve generating contrastive explanations that highlight key differences between possible actions, or using external intention predictors to help humans anticipate the AI's goals. The goal is to foster greater trust and collaboration between humans and AI systems, so they can work together more effectively.
Technical Explanation
The paper reviews the literature on the evolution from explainable AI (XAI) to more interactive and collaborative AI systems. It examines how AI can be designed to improve human-AI teaming and trust by providing greater transparency and explanatory capabilities.
Key topics covered include:
- Explanation Generation: Techniques for AI systems to generate contrastive explanations that highlight key differences between possible actions or outcomes.
- Human-Autonomy Teaming: Approaches for incorporating explanations into human-machine interfaces to foster better collaboration and understanding.
- Intention Prediction: Using external intention predictors to help humans anticipate the goals and behavior of AI systems.
The review synthesizes insights from various fields, including human-computer interaction, cognitive science, and AI safety, to inform the design of more transparent and interactive AI technologies.
Critical Analysis
The paper provides a comprehensive overview of the transition from XAI to interactive AI, but it acknowledges several key limitations and areas for further research:
- Scalability: The proposed techniques for explanation generation and intention prediction may not scale well to complex, real-world AI systems with many possible actions and outcomes.
- Evaluation Challenges: Rigorously evaluating the impact of these interactive AI systems on human trust, collaboration, and decision-making is an ongoing challenge.
- Ethical Considerations: The paper does not deeply explore potential ethical pitfalls, such as the risk of AI systems manipulating human decision-making through biased or deceptive explanations.
Further research is needed to address these limitations and ensure that the shift towards more interactive AI systems is done in a responsible and beneficial manner for both humans and machines.
Conclusion
This literature review outlines the promising transition from explainable AI to interactive AI systems that can better collaborate with and support human decision-making. By generating explanations, predicting intentions, and fostering trust and transparency, these AI systems have the potential to unlock new levels of human-AI cooperation and synergy. However, the research also highlights the need to carefully address scalability, evaluation, and ethical challenges to ensure these technologies are developed and deployed responsibly.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
From Explainable to Interactive AI: A Literature Review on Current Trends in Human-AI Interaction
Muhammad Raees, Inge Meijerink, Ioanna Lykourentzou, Vassilis-Javed Khan, Konstantinos Papangelis
0
0
AI systems are increasingly being adopted across various domains and application areas. With this surge, there is a growing research focus and societal concern for actively involving humans in developing, operating, and adopting these systems. Despite this concern, most existing literature on AI and Human-Computer Interaction (HCI) primarily focuses on explaining how AI systems operate and, at times, allowing users to contest AI decisions. Existing studies often overlook more impactful forms of user interaction with AI systems, such as giving users agency beyond contestability and enabling them to adapt and even co-design the AI's internal mechanics. In this survey, we aim to bridge this gap by reviewing the state-of-the-art in Human-Centered AI literature, the domain where AI and HCI studies converge, extending past Explainable and Contestable AI, delving into the Interactive AI and beyond. Our analysis contributes to shaping the trajectory of future Interactive AI design and advocates for a more user-centric approach that provides users with greater agency, fostering not only their understanding of AI's workings but also their active engagement in its development and evolution.
5/27/2024
Deconstructing Human-AI Collaboration: Agency, Interaction, and Adaptation
Steffen Holter, Mennatallah El-Assady
0
0
As full AI-based automation remains out of reach in most real-world applications, the focus has instead shifted to leveraging the strengths of both human and AI agents, creating effective collaborative systems. The rapid advances in this area have yielded increasingly more complex systems and frameworks, while the nuance of their characterization has gotten more vague. Similarly, the existing conceptual models no longer capture the elaborate processes of these systems nor describe the entire scope of their collaboration paradigms. In this paper, we propose a new unified set of dimensions through which to analyze and describe human-AI systems. Our conceptual model is centered around three high-level aspects - agency, interaction, and adaptation - and is developed through a multi-step process. Firstly, an initial design space is proposed by surveying the literature and consolidating existing definitions and conceptual frameworks. Secondly, this model is iteratively refined and validated by conducting semi-structured interviews with nine researchers in this field. Lastly, to illustrate the applicability of our design space, we utilize it to provide a structured description of selected human-AI systems.
4/19/2024
Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems
Rohan Paleja, Michael Munje, Kimberlee Chang, Reed Jensen, Matthew Gombolay
0
0
Collaborative robots and machine learning-based virtual agents are increasingly entering the human workspace with the aim of increasing productivity and enhancing safety. Despite this, we show in a ubiquitous experimental domain, Overcooked-AI, that state-of-the-art techniques for human-machine teaming (HMT), which rely on imitation or reinforcement learning, are brittle and result in a machine agent that aims to decouple the machine and human's actions to act independently rather than in a synergistic fashion. To remedy this deficiency, we develop HMT approaches that enable iterative, mixed-initiative team development allowing end-users to interactively reprogram interpretable AI teammates. Our 50-subject study provides several findings that we summarize into guidelines. While all approaches underperform a simple collaborative heuristic (a critical, negative result for learning-based methods), we find that white-box approaches supported by interactive modification can lead to significant team development, outperforming white-box approaches alone, and black-box approaches are easier to train and result in better HMT performance highlighting a tradeoff between explainability and interactivity versus ease-of-training. Together, these findings present three important directions: 1) Improving the ability to generate collaborative agents with white-box models, 2) Better learning methods to facilitate collaboration rather than individualized coordination, and 3) Mixed-initiative interfaces that enable users, who may vary in ability, to improve collaboration.
6/10/2024
š
Approximating Human Models During Argumentation-based Dialogues
Yinxu Tang, Stylianos Loukas Vasileiou, William Yeoh
0
0
Explainable AI Planning (XAIP) aims to develop AI agents that can effectively explain their decisions and actions to human users, fostering trust and facilitating human-AI collaboration. A key challenge in XAIP is model reconciliation, which seeks to align the mental models of AI agents and humans. While existing approaches often assume a known and deterministic human model, this simplification may not capture the complexities and uncertainties of real-world interactions. In this paper, we propose a novel framework that enables AI agents to learn and update a probabilistic human model through argumentation-based dialogues. Our approach incorporates trust-based and certainty-based update mechanisms, allowing the agent to refine its understanding of the human's mental state based on the human's expressed trust in the agent's arguments and certainty in their own arguments. We employ a probability weighting function inspired by prospect theory to capture the relationship between trust and perceived probability, and use a Bayesian approach to update the agent's probability distribution over possible human models. We conduct a human-subject study to empirically evaluate the effectiveness of our approach in an argumentation scenario, demonstrating its ability to capture the dynamics of human belief formation and adaptation.
5/30/2024