Autonomous vehicle (AV) proponents projected and claimed that AVs will have better safety than human drivers. The argument is based on human crash data caused by risky driving behaviors that software drivers can avoid. Examples are speeding, driving under the influence, distraction and drowsy driving. Risky driving behaviors can contribute to over 50% of human crashes.
This AV safety assertion has little supporting data—so far. It’s very difficult to prove these claims due to limited AV driving data. In the last month, however, two recent studies provided some promising data that shows software drivers have better safety records compared with human drivers. However, this is in limited use cases and operational domains. Yes, much more data is needed, but at least we have a start.
Swiss Re, a leading global reinsurer, and Waymo started a project in September 2022 to use risk-assessment methodologies to evaluate the safety of AVs. Waymo announced the results in a blog on Sept. 6. Swiss Re also released a seven-page report that explains the methodology and includes summary data.
The Waymo–Swiss Re study is both well done and promising for the future of Waymo’s driverless robotaxi service. The methodology makes eminent sense and was conservative when multiple choices were available.
The result of this project is the first defensible study that shows encouraging robotaxi safety data. The methodology can and should be extended as more driverless vehicle-miles–traveled (VMT) data becomes available. The methodology can also be used for robotrucks when driverless data becomes available.
Waymo determines the safety of its driverless robotaxi service by comparing its liability insurance claims data with Swiss Re private passenger vehicle crash rate baselines for Phoenix and San Francisco. Swiss Re uses both mileage- and zip-code–calibrated data. The following is a summary of Swiss Re human crash rate methodology:
Swiss Re used data from over 600,000 claims in Phoenix and San Francisco from 2016 to 2021.
The desired results are crash estimates per million miles of vehicle travel.
The analysis used the average annual VMT per vehicle. VMT per vehicle is estimated monthly and yearly with state or city granularity.
VMT statistics were provided by the Federal Highway Administration (FHWA).
Included were 600,000 human-driver at-fault claims in Phoenix and San Francisco.
This resulted in the total driving distance surpassing 125 billion miles.
Swiss Re created two VMT per vehicle estimates: one estimate using state VMT data and another based on VMT data per urbanized area.
Swiss Re used state VMT data because it yielded a more conservative and lower baseline crash frequency. State average annual miles per vehicle were higher than in urbanized areas.
Liability insurance claims offer a more comprehensive assessment than collision databases from police-reported crashes because:
Insurance claims data have more consistent standards for reporting.
Claims data are demonstrated to have a higher reporting frequency of safety-relevant crashes.
Police reports don’t capture non-collision-related injuries, and they capture fewer instances of injury claims.
Liability claims data capture information regarding crash or injury causation contribution, as collision responsibility is directly determined during the liability claims settlement process under insurance industry best practices.
The study analyzed claims filed under third-party liability insurance policies, which drivers are required to carry by law in California and Arizona, split by property damage liability and bodily injury liability coverages.
The goal of Swiss Re’s research is to assess how Waymo’s robotaxi service compares with human drivers in terms of safety, which is measured by crash rates calculated by insurance claims and estimated miles traveled.
Waymo’s miles driven from January 2018 to early August 2023 were provided in three categories:
Crash locations were specified by zip codes in Phoenix and San Francisco.
The Waymo Driver’s driverless distance totaled 3.87 million miles.
An additional 35.23 million miles were compiled with the Waymo Driver and safety drivers (thus, the total Waymo Driver miles add up to 39.1 million miles)
Crash rate comparisons were estimated for all categories.
The figure below shows how the Waymo Driver’s crash rates compare with human drivers in the same zip codes in Phoenix and San Francisco. The crash rate is expressed as the number of insurance claims per million miles of travel. The left side shows bodily injuries, with property damage shown on the right.
Waymo Driver vs. human driver crash data (Source: Waymo)
By August 2023, in terms of bodily injuries, the driverless operation of the Waymo Driver didn’t have any crashes with anyone being injured. The Waymo Driver has traveled 3.87 million miles without an injury crash. Bodily injuries by human drivers happened at 1.1 times per million miles driven in the Phoenix and San Francisco areas.
Waymo’s driverless VMT without injury is more than 3.5× the average distance between human crashes with injuries. This is promising, but much higher driverless VMT is needed until the data is truly indicative of self-driving robotaxi performance.
The Waymo Driver VMT with safety driver had a positive injury claim rate at 0.09 per million VMT, compared with human drivers at 1.09 per million VMT. The Waymo improvement ratio is 12.1 versus human drivers.
The combined claims rate of driverless and safety driver VMT was very good, at 0.08 per million VMT, versus 1.09 per million miles for human drivers. The improvement ratio is higher, at 13.6.
Property damage claims in Waymo’s driverless operation were 0.78 claims per million VMT, compared with 3.26 for human drivers. The Waymo improvement ratio was 4.2. It’s interesting that this improvement rate is the lowest for any of the six estimates involving the Waymo Driver.
The Waymo Driver with safety drivers had a claims frequency of 0.17 per million VMT, compared with 3.17 for human drivers in Phoenix and San Francisco. This is an improvement of 18.6 for Waymo.
The combined Waymo Driver claims rate for driverless and with safety drivers was 0.23 per million VMT, versus 3.17 for human drivers—an improvement ratio of 13.7.
The below figure shows how Waymo Driver crash rates declined compared with human drivers in Phoenix and San Francisco. The figure shows the percentage decline for Waymo versus human drivers. The left bar graphs show bodily injuries, with property damage shown on the right.
Waymo Driver vs. human driver crash rate decline data (Source: Waymo)
For bodily injuries, the decline of the driverless operation of the Waymo Driver can’t be calculated, as there have been no bodily injury crashes reported so far. The crash rates for the Waymo Driver with safety drivers, however, declined by 92% compared with human drivers’ bodily injury crash rates.
The combined claims rate of driverless and safety driver VMT declined 93% compared with human drivers.
Property damage claims in Waymo’s driverless operation were reduced by 76% versus human drivers. The Waymo Driver with safety drivers reduced property damage claims by 95%. It’s interesting that the safety driver decline is better than the driverless operation. The combined claims rate of driverless and safety driver VMT for property damage declined by 93% versus human drivers.
The data indicates that Waymo One service is substantially safer than private vehicles driving in Phoenix and San Francisco. The percent reduction in liability claims for the Waymo Driver was in the 90% range in nearly all cases. The safety improvements are based on a large database of high-quality data of private passenger vehicle insurance claims established by Swiss Re, with over 600,000 claims and over 125 billion miles of exposure. The data was calibrated to match Waymo’s mileage distribution across operating locations in Phoenix and San Francisco.
However, we need much more driverless data across many other cities. This Waymo–Swiss Re project has shown a realistic path for compiling and comparing robotaxi safety data and human driver safety performance.
There will undoubtedly be future robotaxi crashes with injuries, and when they happen, there will be negative publicity. A key question is how the public will react to such crashes: Will robotaxis’ better overall crash rates and resulting improved safety be enough to counteract front-page negative news?
