Autonomous Driving with a Deep Dual-Model Solution for Steering and Braking Control

Read original: arXiv:2405.06473 - Published 5/13/2024 by Ana Petra Juki'c, Ana v{S}elek, Marija Seder, Ivana Podnar v{Z}arko

🤿

Overview

This paper presents a deep learning-based solution for autonomous driving that combines steering and braking control using two neural network models.
The steering control is achieved using a modified version of NVIDIA's PilotNet model, which has been optimized for resource-constrained devices.
The braking control is handled by a MobileNet SSD model, which is suitable for devices with limited resources.
The modified PilotNet model achieves similar performance to the original version while significantly reducing the model size and inference latency, making it more suitable for deployment on smaller devices.

Plain English Explanation

Autonomous driving is a rapidly advancing field that has captured the attention of both researchers and industry professionals. This paper presents a deep learning-based solution that aims to address the challenges of autonomous driving by combining steering and braking control using two separate neural network models.

The first model, based on NVIDIA's PilotNet, is responsible for predicting the steering wheel angle required for navigating the vehicle. The researchers recognized that the original PilotNet model might struggle to run efficiently on devices with limited resources, such as smartphones or embedded systems. To address this, they developed a modified version of the PilotNet model that significantly reduces the number of parameters and the memory footprint, while maintaining similar levels of accuracy and performance.

The second model, based on MobileNet SSD, is responsible for controlling the vehicle's braking. This model is designed to be more suitable for resource-constrained devices, as it can operate efficiently on smaller devices with limited processing power and memory.

By combining the strengths of these two models, the researchers have created a comprehensive autonomous driving solution that can effectively control both steering and braking, providing a more complete and robust system. The modified PilotNet model, in particular, represents an important advancement, as it enables the deployment of high-performance steering control on a wider range of devices, including those with limited resources.

Technical Explanation

The core of the autonomous driving solution presented in this paper is the use of two deep neural network models: one for steering control and one for braking control.

For steering control, the researchers used NVIDIA's PilotNet model, which has been widely adopted in the field of autonomous driving. However, they recognized that the original PilotNet model might not be suitable for deployment on resource-constrained devices due to its large model size and high inference latency. To address this, the researchers developed a modified version of the PilotNet model, reducing the number of model parameters and the memory footprint by approximately 60%. This optimization process allowed them to maintain similar levels of loss and accuracy compared to the original PilotNet model, while significantly improving the model's suitability for resource-constrained devices.

For braking control, the researchers used the MobileNet SSD model, which is known for its efficiency and suitability for devices with limited resources. This model is responsible for detecting obstacles and triggering the appropriate braking response to ensure safe operation of the autonomous vehicle.

The performance of the combined steering and braking control system was evaluated in a simulated environment. The results showed that the autonomous driving system using the modified PilotNet model for steering control achieved similar levels of performance compared to the system using the original PilotNet model. This demonstrates the effectiveness of the researchers' optimization efforts, which have made the PilotNet model more suitable for deployment on resource-constrained devices without sacrificing its core functionality.

Critical Analysis

The researchers have presented a promising solution for autonomous driving that combines steering and braking control using two separate deep neural network models. The optimization of the PilotNet model is a particularly noteworthy contribution, as it addresses the challenge of deploying high-performance steering control on a wider range of devices, including those with limited resources.

However, the paper does not provide a detailed analysis of the limitations or potential issues with the proposed solution. For example, it would be helpful to understand the performance of the system in more complex or challenging driving scenarios, such as adverse weather conditions or unexpected road hazards. Additionally, the paper does not discuss the potential for edge cases or failure modes, which are crucial considerations for the safety and reliability of autonomous driving systems.

Furthermore, the paper does not explore the potential for further optimizations or the integration of additional sensors or modalities, such as LiDAR or radar, which could potentially enhance the robustness and performance of the autonomous driving system.

Overall, the research presented in this paper represents a valuable contribution to the field of autonomous driving, particularly in the area of resource-constrained deployment. However, a more comprehensive analysis of the system's limitations, potential issues, and opportunities for further improvement would strengthen the research and provide a more complete understanding of the proposed solution.

Conclusion

This paper presents a deep learning-based solution for autonomous driving that combines steering and braking control using two separate neural network models. The key innovation is the optimization of the PilotNet model, which has been modified to reduce the model size and inference latency, making it more suitable for deployment on resource-constrained devices.

The results demonstrate that the modified PilotNet model achieves similar performance to the original version, while significantly reducing the model's footprint and latency. This advancement is particularly important for enabling the deployment of high-performance steering control on a wider range of devices, including smartphones and embedded systems.

The combination of the modified PilotNet model for steering control and the MobileNet SSD model for braking control provides a comprehensive autonomous driving solution that can effectively navigate and safely operate a vehicle. This research represents an important step forward in the development of autonomous driving systems that can be deployed across a diverse range of hardware platforms, paving the way for more widespread adoption and real-world application of this transformative technology.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

🤿

Autonomous Driving with a Deep Dual-Model Solution for Steering and Braking Control

Ana Petra Juki'c, Ana v{S}elek, Marija Seder, Ivana Podnar v{Z}arko

The technology of autonomous driving is currently attracting a great deal of interest in both research and industry. In this paper, we present a deep learning dual-model solution that uses two deep neural networks for combined braking and steering in autonomous vehicles. Steering control is achieved by applying the NVIDIA's PilotNet model to predict the steering wheel angle, while braking control relies on the use of MobileNet SSD. Both models rely on a single front-facing camera for image input. The MobileNet SSD model is suitable for devices with constrained resources, whereas PilotNet struggles to operate efficiently on smaller devices with limited resources. To make it suitable for such devices, we modified the PilotNet model using our own original network design and reduced the number of model parameters and its memory footprint by approximately 60%. The inference latency has also been reduced, making the model more suitable to operate on resource-constrained devices. The modified PilotNet model achieves similar loss and accuracy compared to the original PilotNet model. When evaluated in a simulated environment, both autonomous driving systems, one using the modified PilotNet model and the other using the original PilotNet model for steering, show similar levels of autonomous driving performance.

5/13/2024

GAD-Generative Learning for HD Map-Free Autonomous Driving

Weijian Sun, Yanbo Jia, Qi Zeng, Zihao Liu, Jiang Liao, Yue Li, Xianfeng Li

Deep-learning-based techniques have been widely adopted for autonomous driving software stacks for mass production in recent years, focusing primarily on perception modules, with some work extending this method to prediction modules. However, the downstream planning and control modules are still designed with hefty handcrafted rules, dominated by optimization-based methods such as quadratic programming or model predictive control. This results in a performance bottleneck for autonomous driving systems in that corner cases simply cannot be solved by enumerating hand-crafted rules. We present a deep-learning-based approach that brings prediction, decision, and planning modules together with the attempt to overcome the rule-based methods' deficiency in real-world applications of autonomous driving, especially for urban scenes. The DNN model we proposed is solely trained with 10 hours of human driver data, and it supports all mass-production ADAS features available on the market to date. This method is deployed onto a Jiyue test car with no modification to its factory-ready sensor set and compute platform. the feasibility, usability, and commercial potential are demonstrated in this article.

6/3/2024

🔮

Rethinking the Integration of Prediction and Planning in Deep Learning-Based Automated Driving Systems: A Review

Steffen Hagedorn, Marcel Hallgarten, Martin Stoll, Alexandru Condurache

Automated driving has the potential to revolutionize personal, public, and freight mobility. Beside accurately perceiving the environment, automated vehicles must plan a safe, comfortable, and efficient motion trajectory. To promote safety and progress, many works rely on modules that predict the future motion of surrounding traffic. Modular automated driving systems commonly handle prediction and planning as sequential, separate tasks. While this accounts for the influence of surrounding traffic on the ego vehicle, it fails to anticipate the reactions of traffic participants to the ego vehicle's behavior. Recent methods increasingly integrate prediction and planning in a joint or interdependent step to model bidirectional interactions. To date, a comprehensive overview of different integration principles is lacking. We systematically review state-of-the-art deep learning-based planning systems, and focus on how they integrate prediction. Different facets of the integration ranging from system architecture to high-level behavioral aspects are considered and related to each other. Moreover, we discuss the implications, strengths, and limitations of different integration principles. By pointing out research gaps, describing relevant future challenges, and highlighting trends in the research field, we identify promising directions for future research.

9/12/2024

Human Insights Driven Latent Space for Different Driving Perspectives: A Unified Encoder for Efficient Multi-Task Inference

Huy-Dung Nguyen, Anass Bairouk, Mirjana Maras, Wei Xiao, Tsun-Hsuan Wang, Patrick Chareyre, Ramin Hasani, Marc Blanchon, Daniela Rus

Autonomous driving holds great potential to transform road safety and traffic efficiency by minimizing human error and reducing congestion. A key challenge in realizing this potential is the accurate estimation of steering angles, which is essential for effective vehicle navigation and control. Recent breakthroughs in deep learning have made it possible to estimate steering angles directly from raw camera inputs. However, the limited available navigation data can hinder optimal feature learning, impacting the system's performance in complex driving scenarios. In this paper, we propose a shared encoder trained on multiple computer vision tasks critical for urban navigation, such as depth, pose, and 3D scene flow estimation, as well as semantic, instance, panoptic, and motion segmentation. By incorporating diverse visual information used by humans during navigation, this unified encoder might enhance steering angle estimation. To achieve effective multi-task learning within a single encoder, we introduce a multi-scale feature network for pose estimation to improve depth learning. Additionally, we employ knowledge distillation from a multi-backbone model pretrained on these navigation tasks to stabilize training and boost performance. Our findings demonstrate that a shared backbone trained on diverse visual tasks is capable of providing overall perception capabilities. While our performance in steering angle estimation is comparable to existing methods, the integration of human-like perception through multi-task learning holds significant potential for advancing autonomous driving systems. More details and the pretrained model are available at https://hi-computervision.github.io/uni-encoder/.

9/17/2024