An Approach to Systematic Data Acquisition and Data-Driven Simulation for the Safety Testing of Automated Driving Functions
2405.01776
0
0
Abstract
With growing complexity and criticality of automated driving functions in road traffic and their operational design domains (ODD), there is increasing demand for covering significant proportions of development, validation, and verification in virtual environments and through simulation models. If, however, simulations are meant not only to augment real-world experiments, but to replace them, quantitative approaches are required that measure to what degree and under which preconditions simulation models adequately represent reality, and thus, using their results accordingly. Especially in R&D areas related to the safety impact of the open world, there is a significant shortage of real-world data to parameterize and/or validate simulations - especially with respect to the behavior of human traffic participants, whom automated driving functions will meet in mixed traffic. We present an approach to systematically acquire data in public traffic by heterogeneous means, transform it into a unified representation, and use it to automatically parameterize traffic behavior models for use in data-driven virtual validation of automated driving functions.
Create account to get full access
Overview
- Presents an approach for systematic data acquisition and data-driven simulation to improve the safety testing of automated driving functions
- Focuses on developing a comprehensive dataset of driving scenarios and using it to train simulation models for safety testing
- Aims to address the challenge of safely testing autonomous vehicles in the real world
Plain English Explanation
This paper describes a method for gathering and using driving data to better test the safety of autonomous vehicle systems. The researchers recognized that it's difficult and dangerous to test self-driving cars in real traffic, so they developed a way to collect a wide variety of driving scenarios in a systematic way. This allows them to create simulation models that can realistically mimic real-world driving conditions. By training autonomous driving systems on these simulated scenarios, the researchers can thoroughly evaluate their safety and performance without the risks of on-road testing. [The goal is to create a comprehensive dataset of driving situations that can be used to train AI models and evaluate the "scenario capability" of self-driving systems. This approach aims to make the development and safety testing of autonomous vehicles more systematic and effective.
Technical Explanation
The paper proposes a framework for systematic data acquisition and data-driven simulation to support the safety testing of automated driving functions. The key elements include:
-
Data Acquisition: The researchers developed a customized data collection platform to gather driving data from a variety of scenarios, including normal driving as well as edge cases and critical situations. This data is used to build a comprehensive dataset of diverse driving situations.
-
Scenario Modeling: The researchers used statistical modeling techniques like Gaussian processes to extract representative driving scenarios from the collected data. These scenarios can then be used to create realistic simulation models.
-
Simulation-based Safety Testing: The paper describes how the simulated driving scenarios can be used to rigorously test the safety and performance of autonomous driving systems in a controlled environment. This allows for efficient exploration of a wide range of conditions without the risks of real-world testing.
The key innovation is the systematic approach to data collection and scenario modeling, which enables the development of realistic simulation environments for comprehensive safety evaluation of autonomous driving functions.
Critical Analysis
The paper presents a thoughtful and comprehensive approach to addressing the challenges of safely testing autonomous vehicles. By focusing on data-driven simulation, the researchers aim to create a more systematic and scalable method for evaluating self-driving systems compared to relying solely on real-world testing.
However, the paper also acknowledges several limitations and areas for future work. For example, the simulation models may not fully capture all the complexities and uncertainties of real-world driving, and there are open questions about how to best translate simulation results to on-road performance. Additionally, the paper does not delve into the ethical and societal implications of automated driving systems, which will be crucial considerations as this technology continues to develop.
Ultimately, this research represents an important step towards safer and more reliable autonomous vehicles, but continued innovation and careful scrutiny will be necessary to address the remaining challenges and ensure these systems are deployed responsibly.
Conclusion
This paper outlines a systematic approach to data acquisition and data-driven simulation for the safety testing of autonomous driving functions. By creating a comprehensive dataset of diverse driving scenarios and using it to train realistic simulation models, the researchers aim to enable more efficient and effective safety evaluation of self-driving systems. This work lays the foundation for further advancements in the field of autonomous vehicle development and deployment, with the potential to improve safety and public trust in this transformative technology.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🧪
A Joint Approach Towards Data-Driven Virtual Testing for Automated Driving: The AVEAS Project
Leon Eisemann, Mirjam Fehling-Kaschek, Silke Forkert, Andreas Forster, Henrik Gommel, Susanne Guenther, Stephan Hammer, David Hermann, Marvin Klemp, Benjamin Lickert, Florian Luettner, Robin Moss, Nicole Neis, Maria Pohle, Dominik Schreiber, Cathrina Sowa, Daniel Stadler, Janina Stompe, Michael Strobelt, David Unger, Jens Ziehn
0
0
With growing complexity and responsibility of automated driving functions in road traffic and growing scope of their operational design domains, there is increasing demand for covering significant parts of development, validation, and verification via virtual environments and simulation models. If, however, simulations are meant not only to augment real-world experiments, but to replace them, quantitative approaches are required that measure to what degree and under which preconditions simulation models adequately represent reality, and thus allow their usage for virtual testing of driving functions. Especially in research and development areas related to the safety impacts of the open world, there is a significant shortage of real-world data to parametrize and/or validate simulations - especially with respect to the behavior of human traffic participants, whom automated vehicles will meet in mixed traffic. This paper presents the intermediate results of the German AVEAS research project (www.aveas.org) which aims at developing methods and metrics for the harmonized, systematic, and scalable acquisition of real-world data for virtual verification and validation of advanced driver assistance systems and automated driving, and establishing an online database following the FAIR principles.
5/13/2024
📊
Statistical Modelling of Driving Scenarios in Road Traffic using Fleet Data of Production Vehicles
Christian Reichenbacher, Jochen Hipp, Oliver Bringmann
0
0
Ensuring the safety of road vehicles at an acceptable level requires the absence of any unreasonable risk arising from all potential hazards linked to the intended au-tomated driving function and its implementation. The assurance that there are no unreasonable risks stemming from hazardous behaviours associated to functional insufficiencies is denoted as safety of intended functionality (SOTIF), a concept outlined in the ISO 21448 standard. In this context, the acquisition of real driving data is considered essential for the verification and validation. For this purpose, we are currently developing a method with which data collect-ed representatively from production vehicles can be modelled into a knowledge-based system in the future. A system that represents the probabilities of occur-rence of concrete driving scenarios over the statistical population of road traffic and makes them usable. The method includes the qualitative and quantitative ab-straction of the drives recorded by the sensors in the vehicles, the possibility of subsequent wireless transmission of the abstracted data from the vehicles and the derivation of the distributions and correlations of scenario parameters. This paper provides a summary of the research project and outlines its central idea. To this end, among other things, the needs for statistical information and da-ta from road traffic are elaborated from ISO 21448, the current state of research is addressed, and methodical aspects are discussed.
4/10/2024
SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries
Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker
0
0
Evaluating the performance of autonomous vehicle planning algorithms necessitates simulating long-tail safety-critical traffic scenarios. However, traditional methods for generating such scenarios often fall short in terms of controllability and realism and neglect the dynamics of agent interactions. To mitigate these limitations, we introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) the generation of realistic long-tail safety-critical scenarios that closely emulate real-world conditions, and 2) enhanced controllability, enabling more comprehensive and interactive evaluations. We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process, which allows an adversarial agent to challenge a planner with plausible maneuvers while all agents in the scene exhibit reactive and realistic behaviors. Furthermore, we propose novel guidance objectives and a partial diffusion process that enables a user to control key aspects of the generated scenarios, such as the collision type and aggressiveness of the adversarial driver, while maintaining the realism of the behavior. We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability. These findings affirm that diffusion models provide a robust and versatile foundation for safety-critical, interactive traffic simulation, extending their utility across the broader landscape of autonomous driving. For supplementary videos, visit our project at https://safe-sim.github.io/.
6/18/2024
🏅
Diagnosing and Predicting Autonomous Vehicle Operational Safety Using Multiple Simulation Modalities and a Virtual Environment
Joe Beck, Shean Huff, Subhadeep Chakraborty
0
0
Even as technology and performance gains are made in the sphere of automated driving, safety concerns remain. Vehicle simulation has long been seen as a tool to overcome the cost associated with a massive amount of on-road testing for development and discovery of safety critical edge-cases. However, purely software-based vehicle models may leave a large realism gap between their real-world counterparts in terms of dynamic response, and highly realistic vehicle-in-the-loop (VIL) simulations that encapsulate a virtual world around a physical vehicle may still be quite expensive to produce and similarly time intensive as on-road testing. In this work, we demonstrate an AV simulation test bed that combines the realism of vehicle-in-the-loop (VIL) simulation with the ease of implementation of model-in-the-loop (MIL) simulation. The setup demonstrated in this work allows for response diagnosis for the VIL simulations. By observing causal links between virtual weather and lighting conditions that surround the virtual depiction of our vehicle, the vision-based perception model and controller of Openpilot, and the dynamic response of our physical vehicle under test, we can draw conclusions regarding how the perceived environment contributed to vehicle response. Conversely, we also demonstrate response prediction for the MIL setup, where the need for a physical vehicle is not required to draw richer conclusions around the impact of environmental conditions on AV performance than could be obtained with VIL simulation alone. These combine for a simulation setup with accurate real-world implications for edge-case discovery that is both cost effective and time efficient to implement.
5/14/2024