Metaverse for Safer Roadways: An Immersive Digital Twin Framework for Exploring Human-Autonomy Coexistence in Urban Transportation Systems
2406.05465
0
0
Abstract
Societal-scale deployment of autonomous vehicles requires them to coexist with human drivers, necessitating mutual understanding and coordination among these entities. However, purely real-world or simulation-based experiments cannot be employed to explore such complex interactions due to safety and reliability concerns, respectively. Consequently, this work presents an immersive digital twin framework to explore and experiment with the interaction dynamics between autonomous and non-autonomous traffic participants. Particularly, we employ a mixed-reality human-machine interface to allow human drivers and autonomous agents to observe and interact with each other for testing edge-case scenarios while ensuring safety at all times. To validate the versatility of the proposed framework's modular architecture, we first present a discussion on a set of user experience experiments encompassing 4 different levels of immersion with 4 distinct user interfaces. We then present a case study of uncontrolled intersection traversal to demonstrate the efficacy of the proposed framework in validating the interactions of a primary human-driven, autonomous, and connected autonomous vehicle with a secondary semi-autonomous vehicle. The proposed framework has been openly released to guide the future of autonomy-oriented digital twins and research on human-autonomy coexistence.
Create account to get full access
Overview
- Presents a framework for creating an immersive digital twin of urban transportation systems to explore the coexistence of humans and autonomous vehicles
- Leverages mixed reality and real-to-simulation (Real2Sim) techniques to bridge the gap between the physical and virtual worlds
- Aims to enhance the safety and efficiency of future transportation systems by enabling the study of human-autonomy interactions in a safe, controlled environment
Plain English Explanation
This research proposes a framework for building an immersive digital twin of urban transportation systems. The goal is to better understand how humans and autonomous vehicles can safely coexist on the roads of the future.
The researchers use a combination of mixed reality and a process called "Real2Sim" to bridge the gap between the real world and a virtual simulation. This allows them to create a highly realistic digital twin that closely mirrors the physical world.
By studying how people and self-driving cars interact in this virtual environment, the researchers hope to uncover insights that can help make future transportation systems safer and more efficient. The digital twin provides a safe, controlled setting to explore these complex human-autonomy interactions without the risks of testing on actual roads.
Technical Explanation
The proposed framework leverages advanced framework for ultra-realistic simulation & digital twinning and towards validation of autonomous vehicles across scales using techniques to create a highly immersive digital twin of urban transportation systems. This digital twin combines real-world data, mixed reality interfaces, and real-to-simulation (Real2Sim) methods to bridge the gap between the physical and virtual environments.
The framework includes several key components:
- Real-world data collection: Sensors and other data sources are used to capture detailed information about the physical transportation system, including vehicle movements, pedestrian behavior, and infrastructure.
- Simulation and modeling: This real-world data is then used to build accurate virtual models of the transportation system, allowing for the simulation of various scenarios and conditions.
- Mixed reality interfaces: Users, including human drivers and autonomous vehicle systems, can interact with the virtual environment through mixed reality headsets and other immersive interfaces, enabling the study of human-autonomy interactions.
- Real2Sim integration: The framework seamlessly integrates real-world data and virtual simulations, allowing for continuous feedback and refinement of the digital twin to maintain its fidelity to the physical system.
By creating this highly realistic and interactive digital twin, the researchers aim to diagnose and predict autonomous vehicle operational safety using and human-machine interaction in automated vehicles for reducing voluntary behaviors. This enables the exploration of complex, emergent behaviors that may arise from the coexistence of humans and autonomous vehicles in urban transportation systems.
Critical Analysis
The proposed framework addresses an important challenge in the field of autonomous vehicles and transportation systems: understanding how humans and autonomous vehicles can safely coexist. By creating a highly realistic digital twin, the researchers aim to provide a safe and controlled environment for exploring these complex interactions.
One potential limitation of the research is the reliance on real-world data collection, which may be subject to biases or incomplete information. Additionally, while the Real2Sim integration aims to maintain the fidelity of the digital twin, there may be inherent differences between the virtual and physical environments that could impact the validity of the findings.
Furthermore, the research does not address the potential low-fidelity digital twin for automated driving systems that may be more practical for certain applications or resource-constrained scenarios. Exploring the trade-offs between fidelity and practicality could provide valuable insights for the broader adoption of such digital twin frameworks.
Conclusion
This research presents a compelling framework for creating an immersive digital twin of urban transportation systems to study the coexistence of humans and autonomous vehicles. By leveraging mixed reality and Real2Sim techniques, the proposed approach aims to bridge the gap between the physical and virtual worlds, enabling the safe and controlled exploration of complex human-autonomy interactions.
The development of such digital twin frameworks has the potential to significantly enhance the safety and efficiency of future transportation systems, as researchers and policymakers can use these virtual environments to test and refine autonomous vehicle technologies and related policies before deploying them in the real world. As the field of autonomous vehicles continues to evolve, this type of integrated simulation and mixed reality platform could become a crucial tool for shaping the future of urban mobility.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Towards Validation of Autonomous Vehicles Across Scales using an Integrated Digital Twin Framework
Tanmay Vilas Samak, Chinmay Vilas Samak, Venkat Narayan Krovi
0
0
Autonomous vehicle platforms of varying spatial scales are employed within the research and development spectrum based on space, safety and monetary constraints. However, deploying and validating autonomy algorithms across varying operational scales presents challenges due to scale-specific dynamics, sensor integration complexities, computational constraints, regulatory considerations, environmental variability, interaction with other traffic participants and scalability concerns. In such a milieu, this work focuses on developing a unified framework for modeling and simulating digital twins of autonomous vehicle platforms across different scales and operational design domains (ODDs) to help support the streamlined development and validation of autonomy software stacks. Particularly, this work discusses the development of digital twin representations of 4 autonomous ground vehicles, which span across 3 different scales and target 3 distinct ODDs. We study the adoption of these autonomy-oriented digital twins to deploy a common autonomy software stack with an aim of end-to-end map-based navigation to achieve the ODD-specific objective(s) for each vehicle. Finally, we also discuss the flexibility of the proposed framework to support virtual, hybrid as well as physical testing with seamless sim2real transfer.
5/8/2024
šÆ
An Advanced Framework for Ultra-Realistic Simulation and Digital Twinning for Autonomous Vehicles
Yuankai He, Hanlin Chen, Weisong Shi
0
0
Simulation is a fundamental tool in developing autonomous vehicles, enabling rigorous testing without the logistical and safety challenges associated with real-world trials. As autonomous vehicle technologies evolve and public safety demands increase, advanced, realistic simulation frameworks are critical. Current testing paradigms employ a mix of general-purpose and specialized simulators, such as CARLA and IVRESS, to achieve high-fidelity results. However, these tools often struggle with compatibility due to differing platform, hardware, and software requirements, severely hampering their combined effectiveness. This paper introduces BlueICE, an advanced framework for ultra-realistic simulation and digital twinning, to address these challenges. BlueICE's innovative architecture allows for the decoupling of computing platforms, hardware, and software dependencies while offering researchers customizable testing environments to meet diverse fidelity needs. Key features include containerization to ensure compatibility across different systems, a unified communication bridge for seamless integration of various simulation tools, and synchronized orchestration of input and output across simulators. This framework facilitates the development of sophisticated digital twins for autonomous vehicle testing and sets a new standard in simulation accuracy and flexibility. The paper further explores the application of BlueICE in two distinct case studies: the ICAT indoor testbed and the STAR campus outdoor testbed at the University of Delaware. These case studies demonstrate BlueICE's capability to create sophisticated digital twins for autonomous vehicle testing and underline its potential as a standardized testbed for future autonomous driving technologies.
5/3/2024
š
Diagnosing and Predicting Autonomous Vehicle Operational Safety Using Multiple Simulation Modalities and a Virtual Environment
Joe Beck, Shean Huff, Subhadeep Chakraborty
0
0
Even as technology and performance gains are made in the sphere of automated driving, safety concerns remain. Vehicle simulation has long been seen as a tool to overcome the cost associated with a massive amount of on-road testing for development and discovery of safety critical edge-cases. However, purely software-based vehicle models may leave a large realism gap between their real-world counterparts in terms of dynamic response, and highly realistic vehicle-in-the-loop (VIL) simulations that encapsulate a virtual world around a physical vehicle may still be quite expensive to produce and similarly time intensive as on-road testing. In this work, we demonstrate an AV simulation test bed that combines the realism of vehicle-in-the-loop (VIL) simulation with the ease of implementation of model-in-the-loop (MIL) simulation. The setup demonstrated in this work allows for response diagnosis for the VIL simulations. By observing causal links between virtual weather and lighting conditions that surround the virtual depiction of our vehicle, the vision-based perception model and controller of Openpilot, and the dynamic response of our physical vehicle under test, we can draw conclusions regarding how the perceived environment contributed to vehicle response. Conversely, we also demonstrate response prediction for the MIL setup, where the need for a physical vehicle is not required to draw richer conclusions around the impact of environmental conditions on AV performance than could be obtained with VIL simulation alone. These combine for a simulation setup with accurate real-world implications for edge-case discovery that is both cost effective and time efficient to implement.
5/14/2024
Designing Interactions With Shared AVs in Complex Urban Mobility Scenarios
Marius Hoggenmueller, Martin Tomitsch, Stewart Worrall
0
0
In this article, we report on the design and evaluation of an external human-machine interface (eHMI) for a real autonomous vehicle (AV), developed to operate as a shared transport pod in a pedestrianized urban space. We present insights about our human-centered design process, which included testing initial concepts through a tangible toolkit and evaluating 360-degree recordings of a staged pick-up scenario in virtual reality. Our results indicate that in complex mobility scenarios, participants filter for critical eHMI messages; further, we found that implicit cues (i.e., pick-up manoeuvre and proximity to the rider) influence participants' experience and trust, while at the same time more explicit interaction modes are desired. This highlights the importance of considering interactions with shared AVs as a service more holistically, in order to develop knowledge about AV-pedestrian interactions in complex mobility scenarios that complements more targeted eHMI evaluations.
6/19/2024