Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles

Read original: arXiv:2312.12675 - Published 7/25/2024 by Kristofer D. Kusano, John M. Scanlon, Yin-Hsiu Chen, Timothy L. McMurry, Ruoshu Chen, Tilia Gode, Trent Victor
Total Score

0

Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • This paper compares the crash rate of Waymo's autonomous driving system (ADS) to human driver benchmarks over 7.1 million miles of operation.
  • The researchers analyzed real-world incident data from Waymo's rider-only ADS to assess its safety performance.
  • The findings provide insights into the potential benefits of autonomous driving technology compared to human drivers.

Plain English Explanation

The paper examines the safety record of Waymo's self-driving car system, which has been tested on over 7 million miles of real-world driving. The researchers looked at the number of crashes or incidents that occurred during this testing period and compared it to the typical rate of crashes for human drivers.

The key finding is that Waymo's autonomous driving system had a significantly lower crash rate than human drivers. This suggests that self-driving cars have the potential to be much safer than cars operated by people. The researchers were able to obtain detailed data on the specific incidents involving Waymo's vehicles, which allowed them to analyze the types of situations where the autonomous system performed better or worse than a human driver.

Overall, this research provides encouraging evidence that autonomous driving technology can enhance road safety compared to relying on human drivers. Of course, the technology still needs further refinement and testing, but these results indicate that self-driving cars could eventually prevent many of the crashes and fatalities that occur on our roads today.

Technical Explanation

The researchers obtained incident data from Waymo's autonomous driving system (ADS) over 7.1 million miles of real-world operation. They analyzed this data to characterize the performance of Waymo's ADS and compare its crash rate to established human driver benchmarks.

The analysis focused on specific types of incidents involving Waymo's rider-only vehicles, such as crashes, near-misses, and disengagements (when the human operator had to take control). The researchers then calculated the equivalent human driver crash rate based on industry standards and government data to provide a direct comparison.

The results showed that Waymo's ADS had a significantly lower crash rate than human drivers, with the autonomous system avoiding many of the types of mistakes and errors that commonly lead to crashes with human operators. This suggests that widespread adoption of autonomous driving technology has the potential to greatly improve overall road safety.

Critical Analysis

The paper provides a rigorous analysis of Waymo's autonomous driving system performance using real-world data over a substantial distance. However, the researchers acknowledge several limitations to their approach, such as the challenges in directly comparing ADS and human driver incident rates given differences in operating environments and data sources.

Additionally, the researchers note that their analysis focuses only on the Waymo ADS and does not necessarily generalize to other autonomous driving systems which may have different capabilities and safety profiles. Further research is needed to assess the performance and safety of a broader range of ADS technologies.

The paper also does not address potential concerns around the ethical and societal implications of autonomous vehicles, such as liability in crash scenarios or the potential displacement of human drivers. These are important considerations that warrant further discussion and analysis.

Overall, this study provides valuable insights into the potential safety benefits of autonomous driving, but additional research and real-world experience will be needed to fully understand the capabilities and tradeoffs of this emerging technology.

Conclusion

This paper presents a detailed analysis comparing the crash rate of Waymo's autonomous driving system to human driver benchmarks over 7.1 million miles of operation. The key finding is that Waymo's ADS demonstrated a significantly lower crash rate than human drivers, suggesting that autonomous vehicles have the potential to greatly enhance road safety.

While the research has some limitations, it provides encouraging evidence that continued advancements in autonomous driving technology could lead to meaningful reductions in crashes, injuries, and fatalities on our roads. As this technology continues to evolve, it will be important to carefully monitor its performance, safety, and broader societal implications.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles
Total Score

0

Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles

Kristofer D. Kusano, John M. Scanlon, Yin-Hsiu Chen, Timothy L. McMurry, Ruoshu Chen, Tilia Gode, Trent Victor

This paper examines the safety performance of the Waymo Driver, an SAE level 4 automated driving system (ADS) used in a rider-only (RO) ride-hailing application without a human driver, either in the vehicle or remotely. ADS crash data was derived from NHTSA's Standing General Order (SGO) reporting over 7.14 million RO miles through the end of October 2023 in Phoenix, AZ, San Francisco, CA, and Los Angeles, CA. When considering all locations together, the any-injury-reported crashed vehicle rate was 0.41 incidents per million miles (IPMM) for the ADS vs 2.80 IPMM for the human benchmark, an 85% reduction or a human crash rate that is 6.7 times higher than the ADS rate. Police-reported crashed vehicle rates for all locations together were 2.1 IPMM for the ADS vs. 4.68 IPMM for the human benchmark, a 55% reduction or a human crash rate that was 2.2 times higher than the ADS rate. Police-reported and any-injury-reported crashed vehicle rate reductions for the ADS were statistically significant when compared in San Francisco and Phoenix, as well as combined across all locations. The any property damage or injury comparison had statistically significant decrease in 3 comparisons, but also non-significant results in 3 other benchmarks. Given imprecision in the benchmark estimate and multiple potential sources of underreporting biasing the benchmarks, caution should be taken when interpreting the results of the any property damage or injury comparison. Together, these crash-rate results should be interpreted as a directional and continuous confidence growth indicator, together with other methodologies, in a safety case approach.

Read more

7/25/2024

📊

Total Score

0

Benchmarks for Retrospective Automated Driving System Crash Rate Analysis Using Police-Reported Crash Data

John M. Scanlon, Kristofer D. Kusano, Laura A. Fraade-Blanar, Timothy L. McMurry, Yin-Hsiu Chen, Trent Victor

With fully automated driving systems (ADS; SAE level 4) ride-hailing services expanding in the US, we are now approaching an inflection point, where the process of retrospectively evaluating ADS safety impact can start to yield statistically credible conclusions. An ADS safety impact measurement requires a comparison to a benchmark crash rate. This study aims to address, update, and extend the existing literature by leveraging police-reported crashes to generate human crash rates for multiple geographic areas with current ADS deployments. All of the data leveraged is publicly accessible, and the benchmark determination methodology is intended to be repeatable and transparent. Generating a benchmark that is comparable to ADS crash data is associated with certain challenges, including data selection, handling underreporting and reporting thresholds, identifying the population of drivers and vehicles to compare against, choosing an appropriate severity level to assess, and matching crash and mileage exposure data. Consequently, we identify essential steps when generating benchmarks, and present our analyses amongst a backdrop of existing ADS benchmark literature. One analysis presented is the usage of established underreporting correction methodology to publicly available human driver police-reported data to improve comparability to publicly available ADS crash data. We also identify important dependencies in controlling for geographic region, road type, and vehicle type, and show how failing to control for these features can bias results. This body of work aims to contribute to the ability of the community - researchers, regulators, industry, and experts - to reach consensus on how to estimate accurate benchmarks.

Read more

7/25/2024

📉

Total Score

0

RAVE Checklist: Recommendations for Overcoming Challenges in Retrospective Safety Studies of Automated Driving Systems

John M. Scanlon, Eric R. Teoh, David G. Kidd, Kristofer D. Kusano, Jonas Bargman, Geoffrey Chi-Johnston, Luigi Di Lillo, Francesca Favaro, Carol Flannagan, Henrik Liers, Bonnie Lin, Magdalena Lindman, Shane McLaughlin, Miguel Perez, Trent Victor

The public, regulators, and domain experts alike seek to understand the effect of deployed SAE level 4 automated driving system (ADS) technologies on safety. The recent expansion of ADS technology deployments is paving the way for early stage safety impact evaluations, whereby the observational data from both an ADS and a representative benchmark fleet are compared to quantify safety performance. In January 2024, a working group of experts across academia, insurance, and industry came together in Washington, DC to discuss the current and future challenges in performing such evaluations. A subset of this working group then met, virtually, on multiple occasions to produce this paper. This paper presents the RAVE (Retrospective Automated Vehicle Evaluation) checklist, a set of fifteen recommendations for performing and evaluating retrospective ADS performance comparisons. The recommendations are centered around the concepts of (1) quality and validity, (2) transparency, and (3) interpretation. Over time, it is anticipated there will be a large and varied body of work evaluating the observed performance of these ADS fleets. Establishing and promoting good scientific practices benefits the work of stakeholders, many of whom may not be subject matter experts. This working group's intentions are to: i) strengthen individual research studies and ii) make the at-large community more informed on how to evaluate this collective body of work.

Read more

8/16/2024

🔗

Total Score

0

Methods to Estimate Advanced Driver Assistance System Penetration Rates in the United States

Noah Goodall

Advanced driver assistance systems (ADAS) are increasingly prevalent in the vehicle fleet, significantly impacting safety and capacity. Transportation agencies struggle to plan for these effects as ADAS availability is not tracked in vehicle registration databases. This paper examines methods to leverage existing public reports and databases to estimate the proportion of vehicles equipped with or utilizing Levels 1 and 2 ADAS technologies in the United States. Findings indicate that in 2022, between 8% and 25% of vehicles were equipped with various ADAS features, though actual usage rates were lower due to driver deactivation. The study proposes strategies to enhance estimates, including analyzing crash data, expanding event data recorder capabilities, conducting naturalistic driving studies, and collaborating with manufacturers to determine installation rates.

Read more

8/6/2024