A Driver-Vehicle Model for ADS Scenario-based Testing
2205.02911
0
0
📈
Abstract
Scenario-based testing for automated driving systems (ADS) must be able to simulate traffic scenarios that rely on interactions with other vehicles. Although many languages for high-level scenario modelling have been proposed, they lack the features to precisely and reliably control the required micro-simulation, while also supporting behavior reuse and test reproducibility for a wide range of interactive scenarios. To fill this gap between scenario design and execution, we propose the Simulated Driver-Vehicle (SDV) model to represent and simulate vehicles as dynamic entities with their behavior being constrained by scenario design and goals set by testers. The model combines driver and vehicle as a single entity. It is based on human-like driving and the mechanical limitations of real vehicles for realistic simulation. The model leverages behavior trees to express high-level behaviors in terms of lower-level maneuvers, affording multiple driving styles and reuse. Furthermore, optimization-based maneuver planners guide the simulated vehicles towards the desired behavior. Our extensive evaluation shows the model's design effectiveness using NHTSA pre-crash scenarios, its motion realism in comparison to naturalistic urban traffic, and its scalability with traffic density. Finally, we show the applicability of our SDV model to test a real ADS and to identify crash scenarios, which are impractical to represent using predefined vehicle trajectories. The SDV model instances can be injected into existing simulation environments via co-simulation.
Create account to get full access
Overview
- The paper proposes a Simulated Driver-Vehicle (SDV) model to represent and simulate vehicles with realistic behavior in scenario-based testing for automated driving systems (ADS).
- The SDV model combines driver and vehicle as a single entity, based on human-like driving and the mechanical limitations of real vehicles.
- The model leverages behavior trees to express high-level behaviors in terms of lower-level maneuvers, allowing for multiple driving styles and reuse.
- Optimization-based maneuver planners guide the simulated vehicles towards the desired behavior.
Plain English Explanation
The paper describes a new way to test automated driving systems (ADSs) by creating realistic simulations of vehicles and their interactions. Traditionally, scenario-based testing for ADSs has relied on predefined vehicle trajectories, which can be limited in their ability to capture the complex interactions that happen in real-world traffic. To address this, the researchers developed the Simulated Driver-Vehicle (SDV) model, which combines the driver and the vehicle into a single entity that behaves in a human-like way, taking into account the physical limitations of the car.
The SDV model uses behavior trees to express high-level driving behaviors, such as changing lanes or merging, in terms of lower-level maneuvers. This allows for the creation of a wide range of driving styles and the reuse of behaviors across different scenarios. Additionally, the model uses optimization-based planners to guide the simulated vehicles towards the desired behavior, ensuring that the simulation is realistic and accurate.
The researchers extensively evaluated the SDV model, showing that it can effectively simulate scenarios from the National Highway Traffic Safety Administration (NHTSA) pre-crash database, match the motion realism of naturalistic urban traffic, and scale to high-density traffic. They also demonstrated how the SDV model can be used to test a real ADS and identify crash scenarios that would be difficult to represent using predefined vehicle trajectories.
Technical Explanation
The paper introduces the Simulated Driver-Vehicle (SDV) model, which aims to address the limitations of existing scenario-based testing approaches for automated driving systems (ADSs). The researchers note that while many languages for high-level scenario modeling have been proposed, they lack the features to precisely and reliably control the required micro-simulation, while also supporting behavior reuse and test reproducibility for a wide range of interactive scenarios.
The SDV model combines the driver and vehicle as a single entity, based on human-like driving and the mechanical limitations of real vehicles. This approach allows for realistic simulation of vehicle behavior, in contrast to the use of predefined vehicle trajectories. The model leverages behavior trees to express high-level behaviors in terms of lower-level maneuvers, enabling the creation of multiple driving styles and the reuse of behaviors across scenarios.
Furthermore, the researchers developed optimization-based maneuver planners to guide the simulated vehicles towards the desired behavior. This ensures that the vehicles exhibit realistic motion and interaction with other traffic participants, as opposed to following predetermined paths.
The paper presents an extensive evaluation of the SDV model, demonstrating its effectiveness in simulating NHTSA pre-crash scenarios, its motion realism compared to naturalistic urban traffic, and its scalability with traffic density. The researchers also show the applicability of the SDV model to test a real ADS and identify crash scenarios that would be impractical to represent using predefined vehicle trajectories.
Finally, the researchers discuss the integration of the SDV model instances into existing simulation environments via co-simulation, enabling the seamless integration of their approach with other tools and frameworks.
Critical Analysis
The paper presents a well-designed and thoroughly evaluated approach to scenario-based testing for automated driving systems (ADSs). The Simulated Driver-Vehicle (SDV) model addresses the limitations of existing scenario modeling languages by providing a more comprehensive and realistic simulation of vehicle behavior and interactions.
One of the key strengths of the SDV model is its use of behavior trees to express high-level driving behaviors in terms of lower-level maneuvers. This approach allows for the creation of a wide range of driving styles and the reuse of behaviors across different scenarios, which is crucial for comprehensive testing of ADSs.
The paper's extensive evaluation, including comparisons to NHTSA pre-crash scenarios and naturalistic urban traffic, provides strong evidence for the model's effectiveness and realism. The researchers' demonstration of the SDV model's applicability to testing a real ADS and identifying crash scenarios that would be difficult to represent using predefined vehicle trajectories further highlights the practical value of their approach.
However, the paper does not address potential limitations or areas for further research in depth. For example, it would be interesting to see how the SDV model performs in scenarios involving complex interactions, such as merge or lane change negotiations, or how it could be extended to incorporate different driver types (e.g., aggressive, distracted) or environmental factors (e.g., weather, infrastructure).
Additionally, while the researchers discuss the integration of the SDV model with existing simulation environments, the paper does not provide detailed information on the computational requirements or scalability of the approach, which could be important considerations for real-world deployment.
Overall, the paper presents a significant contribution to the field of scenario-based testing for ADSs, and the Simulated Driver-Vehicle model offers a promising approach to improving the realism and effectiveness of such testing. Further research and development in this area could lead to even more sophisticated and comprehensive testing frameworks for the safe deployment of autonomous vehicles.
Conclusion
The paper introduces the Simulated Driver-Vehicle (SDV) model, a novel approach to scenario-based testing for automated driving systems (ADSs) that addresses the limitations of existing scenario modeling languages. The SDV model combines the driver and vehicle as a single entity, using behavior trees and optimization-based maneuver planners to enable realistic simulation of vehicle behavior and interactions.
The researchers' extensive evaluation of the SDV model demonstrates its effectiveness in simulating NHTSA pre-crash scenarios, its motion realism compared to naturalistic urban traffic, and its scalability with traffic density. The model's ability to support the testing of real ADS systems and identify complex crash scenarios further highlights its practical value for the development and deployment of autonomous vehicles.
While the paper does not explore all potential limitations or areas for future research, the Simulated Driver-Vehicle model represents a significant contribution to the field of scenario-based testing for ADSs. Continued development and refinement of this approach could lead to even more robust and comprehensive testing frameworks, ultimately supporting the safe and widespread adoption of autonomous vehicle technology.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Towards Interactive Autonomous Vehicle Testing: Vehicle-Under-Test-Centered Traffic Simulation
Yiru Liu, Xiaocong Zhao, Jian Sun
0
0
The simulation-based testing is essential for safely implementing autonomous vehicles (AVs) on roads, necessitating simulated traffic environments that dynamically interact with the Vehicle Under Test (VUT). This study introduces a VUT-Centered environmental Dynamics Inference (VCDI) model for realistic, interactive, and diverse background traffic simulation. VCDI is built on a Transformer-based trajectory inference model to generate trajectories for background objects. Serving the purpose of AV testing, VCDI additionally considers VUT-centered interactivity and scenario diversity using a conditional inference framework. First, the VUT future motion is taken as an augmented model input to bridge the interaction between VUT and background objects. Second, to enrich the scenario diversity, a Bayesian-network-based cost function module is designed. The module, learned in a distributional form, captures the uncertainty of the VUT's strategy, triggering various scenario evolution. Experimental results validate VCDI's trajectory-level simulation precision which outperforms the state-of-the-art trajectory prediction work. The flexibility of the distributional cost function allows VCDI to provide diverse-yet-realistic scenarios for AV testing. We demonstrate such capability by modifying the anticipation to VUT's cost-based strategy and thus achieve multiple testing scenarios with explainable background traffic evolution.
6/6/2024
🏅
Diagnosing and Predicting Autonomous Vehicle Operational Safety Using Multiple Simulation Modalities and a Virtual Environment
Joe Beck, Shean Huff, Subhadeep Chakraborty
0
0
Even as technology and performance gains are made in the sphere of automated driving, safety concerns remain. Vehicle simulation has long been seen as a tool to overcome the cost associated with a massive amount of on-road testing for development and discovery of safety critical edge-cases. However, purely software-based vehicle models may leave a large realism gap between their real-world counterparts in terms of dynamic response, and highly realistic vehicle-in-the-loop (VIL) simulations that encapsulate a virtual world around a physical vehicle may still be quite expensive to produce and similarly time intensive as on-road testing. In this work, we demonstrate an AV simulation test bed that combines the realism of vehicle-in-the-loop (VIL) simulation with the ease of implementation of model-in-the-loop (MIL) simulation. The setup demonstrated in this work allows for response diagnosis for the VIL simulations. By observing causal links between virtual weather and lighting conditions that surround the virtual depiction of our vehicle, the vision-based perception model and controller of Openpilot, and the dynamic response of our physical vehicle under test, we can draw conclusions regarding how the perceived environment contributed to vehicle response. Conversely, we also demonstrate response prediction for the MIL setup, where the need for a physical vehicle is not required to draw richer conclusions around the impact of environmental conditions on AV performance than could be obtained with VIL simulation alone. These combine for a simulation setup with accurate real-world implications for edge-case discovery that is both cost effective and time efficient to implement.
5/14/2024
Model-based generation of representative rear-end crash scenarios across the full severity range using pre-crash data
Jian Wu, Carol Flannagan, Ulrich Sander, Jonas Bargman
0
0
Generating representative rear-end crash scenarios is crucial for safety assessments of Advanced Driver Assistance Systems (ADAS) and Automated Driving systems (ADS). However, existing methods for scenario generation face challenges such as limited and biased in-depth crash data and difficulties in validation. This study sought to overcome these challenges by combining naturalistic driving data and pre-crash kinematics data from rear-end crashes. The combined dataset was weighted to create a representative dataset of rear-end crash characteristics across the full severity range in the United States. Multivariate distribution models were built for the combined dataset, and a driver behavior model for the following vehicle was created by combining two existing models. Simulations were conducted to generate a set of synthetic rear-end crash scenarios, which were then weighted to create a representative synthetic rear-end crash dataset. Finally, the synthetic dataset was validated by comparing the distributions of parameters and the outcomes (Delta-v, the total change in vehicle velocity over the duration of the crash event) of the generated crashes with those in the original combined dataset. The synthetic crash dataset can be used for the safety assessments of ADAS and ADS and as a benchmark when evaluating the representativeness of scenarios generated through other methods.
6/26/2024
Versatile Scene-Consistent Traffic Scenario Generation as Optimization with Diffusion
Zhiyu Huang, Zixu Zhang, Ameya Vaidya, Yuxiao Chen, Chen Lv, Jaime Fern'andez Fisac
0
0
Generating realistic and controllable agent behaviors in traffic simulation is crucial for the development of autonomous vehicles. This problem is often formulated as imitation learning (IL) from real-world driving data by either directly predicting future trajectories or inferring cost functions with inverse optimal control. In this paper, we draw a conceptual connection between IL and diffusion-based generative modeling and introduce a novel framework Versatile Behavior Diffusion (VBD) to simulate interactive scenarios with multiple traffic participants. Our model not only generates scene-consistent multi-agent interactions but also enables scenario editing through multi-step guidance and refinement. Experimental evaluations show that VBD achieves state-of-the-art performance on the Waymo Sim Agents benchmark. In addition, we illustrate the versatility of our model by adapting it to various applications. VBD is capable of producing scenarios conditioning on priors, integrating with model-based optimization, sampling multi-modal scene-consistent scenarios by fusing marginal predictions, and generating safety-critical scenarios when combined with a game-theoretic solver.
4/4/2024