Systematic Review: Anomaly Detection in Connected and Autonomous Vehicles
2405.02731
0
0
❗
Abstract
This systematic review focuses on anomaly detection for connected and autonomous vehicles. The initial database search identified 2160 articles, of which 203 were included in this review after rigorous screening and assessment. This study revealed that the most commonly used Artificial Intelligence (AI) algorithms employed in anomaly detection are neural networks like LSTM, CNN, and autoencoders, alongside one-class SVM. Most anomaly-based models were trained using real-world operational vehicle data, although anomalies, such as attacks and faults, were often injected artificially into the datasets. These models were evaluated mostly using five key evaluation metrics: recall, accuracy, precision, F1-score, and false positive rate. The most frequently used selection of evaluation metrics used for anomaly detection models were accuracy, precision, recall, and F1-score. This systematic review presents several recommendations. First, there is a need to incorporate multiple evaluation metrics to provide a comprehensive assessment of the anomaly detection models. Second, only a small proportion of the studies have made their models open source, indicating a need to share models publicly to facilitate collaboration within the research community, and to validate and compare findings effectively. Third, there is a need for benchmarking datasets with predefined anomalies or cyberattacks to test and improve the effectiveness of the proposed anomaly-based detection models. Furthermore, there is a need for future research to investigate the deployment of anomaly detection to a vehicle to assess its performance on the road. There is a notable lack of research done on intrusion detection systems using different protocols to CAN, such as Ethernet and FlexRay.
Create account to get full access
Overview
- This systematic review focuses on anomaly detection for connected and autonomous vehicles.
- 2160 articles were initially identified, with 203 included in the review after screening.
- The most common AI algorithms used are neural networks like LSTM, CNN, and autoencoders, as well as one-class SVM.
- Most models were trained on real-world vehicle data, with anomalies often injected artificially.
- Evaluation metrics used include recall, accuracy, precision, F1-score, and false positive rate.
Plain English Explanation
This research paper looks at how anomaly detection can be used to identify unusual or problematic behavior in connected and autonomous vehicles. The researchers started by gathering a large number of relevant studies, and then carefully reviewed 203 of them in detail.
The key finding is that artificial intelligence (AI) algorithms, particularly neural networks like Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and autoencoders, are commonly used for anomaly detection in vehicle systems. These models are often trained on real-world vehicle data, with anomalies or problems sometimes added artificially to the datasets to test the models.
To evaluate the performance of these anomaly detection models, the researchers looked at metrics like accuracy, precision, recall, and F1-score. These help measure how well the models can identify abnormal vehicle behavior, while also minimizing false alarms.
Technical Explanation
The systematic review began with a database search that identified 2,160 potentially relevant articles. After a rigorous screening and assessment process, 203 of these studies were included in the final review.
The analysis revealed that the most commonly used AI algorithms for anomaly detection in connected and autonomous vehicles are neural networks, such as LSTM, CNN, and autoencoders. These were often used alongside one-class Support Vector Machines (SVMs).
Most of the anomaly detection models were trained using real-world operational data from vehicles. However, the researchers noted that anomalies, such as cyber attacks or system faults, were frequently injected artificially into the datasets to test the models' performance.
To evaluate the anomaly detection models, the studies primarily used five key metrics: recall, accuracy, precision, F1-score, and false positive rate. The most common combination of evaluation metrics was accuracy, precision, recall, and F1-score.
Critical Analysis
The researchers identified several important recommendations for future work in this area. First, they suggest that incorporating multiple evaluation metrics can provide a more comprehensive assessment of anomaly detection models, rather than relying on a single metric.
Additionally, the researchers noted that only a small proportion of the studies had made their models openly available. Sharing models publicly could facilitate greater collaboration within the research community and allow for more effective validation and comparison of findings.
The paper also highlighted the need for standardized benchmarking datasets that include predefined anomalies or cyber attacks. This could help researchers test and improve the effectiveness of their proposed anomaly detection models.
Furthermore, the researchers identified a lack of research on the deployment of anomaly detection systems directly in vehicles to assess their real-world performance.
Conclusion
This systematic review provides a comprehensive overview of the current state of anomaly detection research for connected and autonomous vehicles. The findings suggest that AI-based models, particularly neural networks, are widely used for this purpose, with a focus on evaluating their performance using metrics like accuracy, precision, and recall.
However, the review also highlights several areas for further development, including the need for more comprehensive evaluation, increased model sharing, standardized benchmarking datasets, and real-world deployment testing. Addressing these gaps could help advance the field of anomaly detection and enhance the safety and security of future autonomous vehicle systems.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
A Framework for the Systematic Assessment of Anomaly Detectors in Time-Sensitive Automotive Networks
Philipp Meyer, Timo Hackel, Teresa Lubeck, Franz Korf, Thomas C. Schmidt
0
0
Connected cars are susceptible to cyberattacks. Security and safety of future vehicles highly depend on a holistic protection of automotive components, of which the time-sensitive backbone network takes a significant role. These onboard Time-Sensitive Networks (TSNs) require monitoring for safety and -- as versatile platforms to host Network Anomaly Detection Systems (NADSs) -- for security. Still a thorough evaluation of anomaly detection methods in the context of hard real-time operations, automotive protocol stacks, and domain specific attack vectors is missing along with appropriate input datasets. In this paper, we present an assessment framework that allows for reproducible, comparable, and rapid evaluation of detection algorithms. It is based on a simulation toolchain, which contributes configurable topologies, traffic streams, anomalies, attacks, and detectors. We demonstrate the assessment of NADSs in a comprehensive in-vehicular network with its communication flows, on which we model traffic anomalies. We evaluate exemplary detection mechanisms and reveal how the detection performance is influenced by different combinations of TSN traffic flows and anomaly types. Our approach translates to other real-time Ethernet domains, such as industrial facilities, airplanes, and UAVs.
5/3/2024
❗
An Anomaly Behavior Analysis Framework for Securing Autonomous Vehicle Perception
Murad Mehrab Abrar, Salim Hariri
0
0
As a rapidly growing cyber-physical platform, Autonomous Vehicles (AVs) are encountering more security challenges as their capabilities continue to expand. In recent years, adversaries are actively targeting the perception sensors of autonomous vehicles with sophisticated attacks that are not easily detected by the vehicles' control systems. This work proposes an Anomaly Behavior Analysis approach to detect a perception sensor attack against an autonomous vehicle. The framework relies on temporal features extracted from a physics-based autonomous vehicle behavior model to capture the normal behavior of vehicular perception in autonomous driving. By employing a combination of model-based techniques and machine learning algorithms, the proposed framework distinguishes between normal and abnormal vehicular perception behavior. To demonstrate the application of the framework in practice, we performed a depth camera attack experiment on an autonomous vehicle testbed and generated an extensive dataset. We validated the effectiveness of the proposed framework using this real-world data and released the dataset for public access. To our knowledge, this dataset is the first of its kind and will serve as a valuable resource for the research community in evaluating their intrusion detection techniques effectively.
4/8/2024
Enhanced Anomaly Detection in Automotive Systems Using SAAD: Statistical Aggregated Anomaly Detection
Dacian Goina, Eduard Hogea, George Maties
0
0
This paper presents a novel anomaly detection methodology termed Statistical Aggregated Anomaly Detection (SAAD). The SAAD approach integrates advanced statistical techniques with machine learning, and its efficacy is demonstrated through validation on real sensor data from a Hardware-in-the-Loop (HIL) environment within the automotive domain. The key innovation of SAAD lies in its ability to significantly enhance the accuracy and robustness of anomaly detection when combined with Fully Connected Networks (FCNs) augmented by dropout layers. Comprehensive experimental evaluations indicate that the standalone statistical method achieves an accuracy of 72.1%, whereas the deep learning model alone attains an accuracy of 71.5%. In contrast, the aggregated method achieves a superior accuracy of 88.3% and an F1 score of 0.921, thereby outperforming the individual models. These results underscore the effectiveness of SAAD, demonstrating its potential for broad application in various domains, including automotive systems.
6/14/2024
Hybrid Video Anomaly Detection for Anomalous Scenarios in Autonomous Driving
Daniel Bogdoll, Jan Imhof, Tim Joseph, J. Marius Zollner
0
0
In autonomous driving, the most challenging scenarios are the ones that can only be detected within their temporal context. Most video anomaly detection approaches focus either on surveillance or traffic accidents, which are only a subfield of autonomous driving. In this work, we present HF$^2$-VAD$_{AD}$, a variation of the HF$^2$-VAD surveillance video anomaly detection method for autonomous driving. We learn a representation of normality from a vehicle's ego perspective and evaluate pixel-wise anomaly detections in rare and critical scenarios.
6/11/2024