Uncertainty-Aware DRL for Autonomous Vehicle Crowd Navigation in Shared Space
2405.13969
0
0
Abstract
Safe, socially compliant, and efficient navigation of low-speed autonomous vehicles (AVs) in pedestrian-rich environments necessitates considering pedestrians' future positions and interactions with the vehicle and others. Despite the inevitable uncertainties associated with pedestrians' predicted trajectories due to their unobserved states (e.g., intent), existing deep reinforcement learning (DRL) algorithms for crowd navigation often neglect these uncertainties when using predicted trajectories to guide policy learning. This omission limits the usability of predictions when diverging from ground truth. This work introduces an integrated prediction and planning approach that incorporates the uncertainties of predicted pedestrian states in the training of a model-free DRL algorithm. A novel reward function encourages the AV to respect pedestrians' personal space, decrease speed during close approaches, and minimize the collision probability with their predicted paths. Unlike previous DRL methods, our model, designed for AV operation in crowded spaces, is trained in a novel simulation environment that reflects realistic pedestrian behaviour in a shared space with vehicles. Results show a 40% decrease in collision rate and a 15% increase in minimum distance to pedestrians compared to the state of the art model that does not account for prediction uncertainty. Additionally, the approach outperforms model predictive control methods that incorporate the same prediction uncertainties in terms of both performance and computational time, while producing trajectories closer to human drivers in similar scenarios.
Create account to get full access
Overview
- Presents an uncertainty-aware deep reinforcement learning (DRL) approach for autonomous vehicle navigation in shared spaces with pedestrians
- Focuses on improving safety and efficiency of autonomous vehicle interactions with pedestrians
- Incorporates uncertainty modeling to account for unpredictable pedestrian behavior
Plain English Explanation
This research paper explores a new way for self-driving cars to navigate crowded areas, like city streets, while safely interacting with pedestrians. Traditional self-driving systems often struggle with the unpredictable movements of people walking around. To address this, the researchers developed a deep reinforcement learning (DRL) approach that can better account for the uncertainty in pedestrian behavior.
The key idea is to model the uncertainty in how pedestrians might move, rather than just trying to predict their exact path. This allows the self-driving car to plan its actions in a way that is robust to unexpected pedestrian movements. The system learns through trial-and-error interactions, similar to how a human driver would, but with the added benefit of being able to consider a range of possible futures.
By incorporating this uncertainty modeling, the self-driving car can make safer and more efficient decisions when navigating crowded environments with pedestrians. This could lead to more widespread adoption of autonomous vehicles, as it helps address a major challenge in real-world deployment.
Technical Explanation
The paper proposes an "Uncertainty-Aware DRL for Autonomous Vehicle Crowd Navigation in Shared Space" approach that combines deep reinforcement learning with explicit modeling of uncertainty in pedestrian behavior. This builds on prior work in coupled prediction and planning and evaluating pedestrian trajectory prediction methods.
The key components of the system include:
- A Memory-Enhanced Deep Reinforcement Learning (MESA-DRL) agent that learns to navigate the shared space
- A pedestrian trajectory prediction model that outputs a distribution over possible future paths, capturing uncertainty
- A planning module that reasons about the predicted pedestrian trajectories to find safe and efficient paths for the autonomous vehicle
During training, the MESA-DRL agent interacts with a simulation environment containing pedestrians whose movements are modeled probabilistically. The agent learns to balance safety, efficiency, and comfort through reward signals that encourage behaviors like collision avoidance, progress towards the goal, and smooth maneuvers.
The authors evaluate their approach on benchmark datasets and show that it outperforms prior methods in terms of metrics like minimum distance to pedestrians and smoothness of the autonomous vehicle's trajectory. This demonstrates the benefits of the uncertainty-aware design for safe and effective navigation in shared spaces.
Critical Analysis
The paper makes a compelling case for the importance of modeling uncertainty in autonomous vehicle navigation, particularly in crowded urban environments. The proposed MESA-DRL approach appears to be a well-designed and principled solution that builds upon relevant prior work, such as evaluating pedestrian trajectory prediction methods and autonomous vehicle decision control through reinforcement learning.
One potential limitation of the research is the reliance on simulation-based training and evaluation. While this allows for controlled experimentation, it may not fully capture the complexities and unpredictability of real-world pedestrian behavior. Further validation on physical test platforms or in-the-field deployments would strengthen the findings.
Additionally, the paper does not provide much discussion on the computational and memory requirements of the MESA-DRL system, which could be an important practical consideration for deployment on resource-constrained autonomous vehicle hardware.
Overall, this work represents a valuable contribution to the field of autonomous vehicle navigation, particularly in its emphasis on accounting for uncertainty and its potential to improve the safety and acceptance of self-driving cars in shared spaces. Continued research and development in this area could lead to significant advancements in the real-world deployment of autonomous vehicles.
Conclusion
The "Uncertainty-Aware DRL for Autonomous Vehicle Crowd Navigation in Shared Space" paper presents a novel approach to enable safe and efficient autonomous vehicle navigation in crowded environments with pedestrians. By incorporating explicit modeling of uncertainty in pedestrian behavior, the proposed MESA-DRL system can make more informed and robust decisions, leading to improved safety and comfort for both the autonomous vehicle and surrounding pedestrians.
This research represents an important step forward in addressing a key challenge for the widespread adoption of autonomous vehicles. As self-driving cars become more prevalent, it will be essential for them to navigate shared spaces, like city streets, in a way that is responsive to the unpredictable movements of pedestrians. The uncertainty-aware techniques developed in this work could pave the way for more reliable and trustworthy autonomous vehicle systems, ultimately contributing to the goal of safer and more accessible transportation for all.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🔮
Uncertainty-Aware Pedestrian Trajectory Prediction via Distributional Diffusion
Yao Liu, Zesheng Ye, Rui Wang, Binghao Li, Quan Z. Sheng, Lina Yao
0
0
Tremendous efforts have been put forth on predicting pedestrian trajectory with generative models to accommodate uncertainty and multi-modality in human behaviors. An individual's inherent uncertainty, e.g., change of destination, can be masked by complex patterns resulting from the movements of interacting pedestrians. However, latent variable-based generative models often entangle such uncertainty with complexity, leading to limited either latent expressivity or predictive diversity. In this work, we propose to separately model these two factors by implicitly deriving a flexible latent representation to capture intricate pedestrian movements, while integrating predictive uncertainty of individuals with explicit bivariate Gaussian mixture densities over their future locations. More specifically, we present a model-agnostic uncertainty-aware pedestrian trajectory prediction framework, parameterizing sufficient statistics for the mixture of Gaussians that jointly comprise the multi-modal trajectories. We further estimate these parameters of interest by approximating a denoising process that progressively recovers pedestrian movements from noise. Unlike previous studies, we translate the predictive stochasticity to explicit distributions, allowing it to readily generate plausible future trajectories indicating individuals' self-uncertainty. Moreover, our framework is compatible with different neural net architectures. We empirically show the performance gains over state-of-the-art even with lighter backbones, across most scenes on two public benchmarks.
5/14/2024
Structured Graph Network for Constrained Robot Crowd Navigation with Low Fidelity Simulation
Shuijing Liu, Kaiwen Hong, Neeloy Chakraborty, Katherine Driggs-Campbell
0
0
We investigate the feasibility of deploying reinforcement learning (RL) policies for constrained crowd navigation using a low-fidelity simulator. We introduce a representation of the dynamic environment, separating human and obstacle representations. Humans are represented through detected states, while obstacles are represented as computed point clouds based on maps and robot localization. This representation enables RL policies trained in a low-fidelity simulator to deploy in real world with a reduced sim2real gap. Additionally, we propose a spatio-temporal graph to model the interactions between agents and obstacles. Based on the graph, we use attention mechanisms to capture the robot-human, human-human, and human-obstacle interactions. Our method significantly improves navigation performance in both simulated and real-world environments. Video demonstrations can be found at https://sites.google.com/view/constrained-crowdnav/home.
5/29/2024
Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach
Mehran Berahman, Majid Rostami-Shahrbabaki, Klaus Bogenberger
0
0
Deep reinforcement learning has shown promise in various engineering applications, including vehicular traffic control. The non-stationary nature of traffic, especially in the lane-free environment with more degrees of freedom in vehicle behaviors, poses challenges for decision-making since a wrong action might lead to a catastrophic failure. In this paper, we propose a novel driving strategy for Connected and Automated Vehicles (CAVs) based on a competitive Multi-Agent Deep Deterministic Policy Gradient approach. The developed multi-agent deep reinforcement learning algorithm creates a dynamic and non-stationary scenario, mirroring real-world traffic complexities and making trained agents more robust. The algorithm's reward function is strategically and uniquely formulated to cover multiple vehicle control tasks, including maintaining desired speeds, overtaking, collision avoidance, and merging and diverging maneuvers. Moreover, additional considerations for both lateral and longitudinal passenger comfort and safety criteria are taken into account. We employed inter-vehicle forces, known as nudging and repulsive forces, to manage the maneuvers of CAVs in a lane-free traffic environment. The proposed driving algorithm is trained and evaluated on lane-free roads using the Simulation of Urban Mobility platform. Experimental results demonstrate the algorithm's efficacy in handling different objectives, highlighting its potential to enhance safety and efficiency in autonomous driving within lane-free traffic environments.
6/24/2024
QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving
Sourav Biswas, Sergio Casas, Quinlan Sykora, Ben Agro, Abbas Sadat, Raquel Urtasun
0
0
A self-driving vehicle must understand its environment to determine the appropriate action. Traditional autonomy systems rely on object detection to find the agents in the scene. However, object detection assumes a discrete set of objects and loses information about uncertainty, so any errors compound when predicting the future behavior of those agents. Alternatively, dense occupancy grid maps have been utilized to understand free-space. However, predicting a grid for the entire scene is wasteful since only certain spatio-temporal regions are reachable and relevant to the self-driving vehicle. We present a unified, interpretable, and efficient autonomy framework that moves away from cascading modules that first perceive, then predict, and finally plan. Instead, we shift the paradigm to have the planner query occupancy at relevant spatio-temporal points, restricting the computation to those regions of interest. Exploiting this representation, we evaluate candidate trajectories around key factors such as collision avoidance, comfort, and progress for safety and interpretability. Our approach achieves better highway driving quality than the state-of-the-art in high-fidelity closed-loop simulations.
4/3/2024