Tom Fehring
@tom.fehri.ng
Data person. Formerly at OpenAI, even more formerly an actuary. Posts (rarely) about data, statistical computing, ML/AI, financial markets, urbanism, and/or cats.
Note that all of the specifics here apply only to Waymo, not to self-driving cars in general.
I increasingly worry that regulators and the public will paint all platforms with the same brush, and was pleasantly surprised that that didn't seem to happen after the Cruise incident.
19/19
I increasingly worry that regulators and the public will paint all platforms with the same brush, and was pleasantly surprised that that didn't seem to happen after the Cruise incident.
19/19
October 5, 2025 at 9:13 PM
Note that all of the specifics here apply only to Waymo, not to self-driving cars in general.
I increasingly worry that regulators and the public will paint all platforms with the same brush, and was pleasantly surprised that that didn't seem to happen after the Cruise incident.
19/19
I increasingly worry that regulators and the public will paint all platforms with the same brush, and was pleasantly surprised that that didn't seem to happen after the Cruise incident.
19/19
The reason I only say "fairly high" doesn't show up in the data at all - it's ~entirely due to the risk of a bad software update, security incident, etc.
This is unlikely, but it's hard to say exactly how unlikely, and it could be very high-severity if it happened.
18/
This is unlikely, but it's hard to say exactly how unlikely, and it could be very high-severity if it happened.
18/
October 5, 2025 at 9:13 PM
The reason I only say "fairly high" doesn't show up in the data at all - it's ~entirely due to the risk of a bad software update, security incident, etc.
This is unlikely, but it's hard to say exactly how unlikely, and it could be very high-severity if it happened.
18/
This is unlikely, but it's hard to say exactly how unlikely, and it could be very high-severity if it happened.
18/
FWIW, I have fairly high confidence that Waymo is safer than driving or rideshare in all of its established metros, though not by enough to justify the cost from a pure QALY/$ perspective.
(This is based on my experience of Waymo being ~$5-$15 more than Uber/Lyft depending on distance.)
17/
(This is based on my experience of Waymo being ~$5-$15 more than Uber/Lyft depending on distance.)
17/
October 5, 2025 at 9:13 PM
FWIW, I have fairly high confidence that Waymo is safer than driving or rideshare in all of its established metros, though not by enough to justify the cost from a pure QALY/$ perspective.
(This is based on my experience of Waymo being ~$5-$15 more than Uber/Lyft depending on distance.)
17/
(This is based on my experience of Waymo being ~$5-$15 more than Uber/Lyft depending on distance.)
17/
From a measurement perspective, it's fine to use a metric that allocates >0 fatalities to Waymo for each of these incidents.
From a practical perspective, if these incidents cause you to think Waymo is less safe than you previously believed, I think that's the wrong conclusion.
16/
From a practical perspective, if these incidents cause you to think Waymo is less safe than you previously believed, I think that's the wrong conclusion.
16/
October 5, 2025 at 9:13 PM
From a measurement perspective, it's fine to use a metric that allocates >0 fatalities to Waymo for each of these incidents.
From a practical perspective, if these incidents cause you to think Waymo is less safe than you previously believed, I think that's the wrong conclusion.
16/
From a practical perspective, if these incidents cause you to think Waymo is less safe than you previously believed, I think that's the wrong conclusion.
16/
A nice thing about small datasets is that you can just look at the individual datapoints.
In both of the fatal Waymo-involved accidents to date, the Waymo was rear-ended while at a standstill or near-standstill: once in traffic, once while yielding to a pedestrian before a right turn.
15/
In both of the fatal Waymo-involved accidents to date, the Waymo was rear-ended while at a standstill or near-standstill: once in traffic, once while yielding to a pedestrian before a right turn.
15/
October 5, 2025 at 9:13 PM
A nice thing about small datasets is that you can just look at the individual datapoints.
In both of the fatal Waymo-involved accidents to date, the Waymo was rear-ended while at a standstill or near-standstill: once in traffic, once while yielding to a pedestrian before a right turn.
15/
In both of the fatal Waymo-involved accidents to date, the Waymo was rear-ended while at a standstill or near-standstill: once in traffic, once while yielding to a pedestrian before a right turn.
15/
The caveat is that even a perfect driver (which, to be clear, Waymo is not) would be responsible for >0 fatalities in expectation based on a blameless approach.
That doesn't mean it's wrong - just a caveat that needs to be kept in mind.
14/
That doesn't mean it's wrong - just a caveat that needs to be kept in mind.
14/
October 5, 2025 at 9:13 PM
The caveat is that even a perfect driver (which, to be clear, Waymo is not) would be responsible for >0 fatalities in expectation based on a blameless approach.
That doesn't mean it's wrong - just a caveat that needs to be kept in mind.
14/
That doesn't mean it's wrong - just a caveat that needs to be kept in mind.
14/
Now, returning to the 0.47 allocated fatalities
There's a lot to like about the author's methodology, for exactly the reasons he points out. In fact, I'm inclined to agree that blame shouldn't be accounted for in safety metrics.
13/
There's a lot to like about the author's methodology, for exactly the reasons he points out. In fact, I'm inclined to agree that blame shouldn't be accounted for in safety metrics.
13/
October 5, 2025 at 9:13 PM
Now, returning to the 0.47 allocated fatalities
There's a lot to like about the author's methodology, for exactly the reasons he points out. In fact, I'm inclined to agree that blame shouldn't be accounted for in safety metrics.
13/
There's a lot to like about the author's methodology, for exactly the reasons he points out. In fact, I'm inclined to agree that blame shouldn't be accounted for in safety metrics.
13/
You could come up with a hierarchical approach to formalize this, but informally: a sufficiently-powered 79% reduction in crashes plus an underpowered 70% reduction in fatalities together provide practically significant evidence of a reduction in fatalities.
12/
12/
October 5, 2025 at 9:13 PM
You could come up with a hierarchical approach to formalize this, but informally: a sufficiently-powered 79% reduction in crashes plus an underpowered 70% reduction in fatalities together provide practically significant evidence of a reduction in fatalities.
12/
12/
But Waymo's report from this January (through 56.7M VMT, doi.org/10.1080/1538...) consistently shows >70% reductions in crashes.
The headline number is 79% fewer any-injury crashes (yes, regardless of fault). This one is highly stat sig because crashes are much more common than fatalities.
11/
The headline number is 79% fewer any-injury crashes (yes, regardless of fault). This one is highly stat sig because crashes are much more common than fatalities.
11/
October 5, 2025 at 9:13 PM
But Waymo's report from this January (through 56.7M VMT, doi.org/10.1080/1538...) consistently shows >70% reductions in crashes.
The headline number is 79% fewer any-injury crashes (yes, regardless of fault). This one is highly stat sig because crashes are much more common than fatalities.
11/
The headline number is 79% fewer any-injury crashes (yes, regardless of fault). This one is highly stat sig because crashes are much more common than fatalities.
11/
I disagree with the author's claim that this is "close enough to be considered roughly comparable to a human driver."
Now, this difference isn't stat sig. The sample size is underpowered given the low base rate - human drivers would only be expected to incur ~1.57 fatalities in 128M VMT.
10/
Now, this difference isn't stat sig. The sample size is underpowered given the low base rate - human drivers would only be expected to incur ~1.57 fatalities in 128M VMT.
10/
October 5, 2025 at 9:13 PM
I disagree with the author's claim that this is "close enough to be considered roughly comparable to a human driver."
Now, this difference isn't stat sig. The sample size is underpowered given the low base rate - human drivers would only be expected to incur ~1.57 fatalities in 128M VMT.
10/
Now, this difference isn't stat sig. The sample size is underpowered given the low base rate - human drivers would only be expected to incur ~1.57 fatalities in 128M VMT.
10/
So the relevant comparison is:
- Waymo - 0.47 fatalities in 128M VMT, a rate of 0.37/100M
- Human drivers - 97,386 fatalities in 81.4T VMT from 2023 to 1H 2025, a rate of 1.20/100M, or ~1.23/100M when time- and location-matched to Waymo
i.e., Waymo had 70% fewer fatalities than the base rate
9/
- Waymo - 0.47 fatalities in 128M VMT, a rate of 0.37/100M
- Human drivers - 97,386 fatalities in 81.4T VMT from 2023 to 1H 2025, a rate of 1.20/100M, or ~1.23/100M when time- and location-matched to Waymo
i.e., Waymo had 70% fewer fatalities than the base rate
9/
October 5, 2025 at 9:13 PM
So the relevant comparison is:
- Waymo - 0.47 fatalities in 128M VMT, a rate of 0.37/100M
- Human drivers - 97,386 fatalities in 81.4T VMT from 2023 to 1H 2025, a rate of 1.20/100M, or ~1.23/100M when time- and location-matched to Waymo
i.e., Waymo had 70% fewer fatalities than the base rate
9/
- Waymo - 0.47 fatalities in 128M VMT, a rate of 0.37/100M
- Human drivers - 97,386 fatalities in 81.4T VMT from 2023 to 1H 2025, a rate of 1.20/100M, or ~1.23/100M when time- and location-matched to Waymo
i.e., Waymo had 70% fewer fatalities than the base rate
9/
At that rate, Waymo would have been around 122M cumulative VMT on 9/14, the day of the second fatal crash, and 128M today.
That implies a cumulative fatality rate of 0.37 per 100M VMT today, with an all-time-high of 0.39 on 9/14.
8/
That implies a cumulative fatality rate of 0.37 per 100M VMT today, with an all-time-high of 0.39 on 9/14.
8/
October 5, 2025 at 9:13 PM
At that rate, Waymo would have been around 122M cumulative VMT on 9/14, the day of the second fatal crash, and 128M today.
That implies a cumulative fatality rate of 0.37 per 100M VMT today, with an all-time-high of 0.39 on 9/14.
8/
That implies a cumulative fatality rate of 0.37 per 100M VMT today, with an all-time-high of 0.39 on 9/14.
8/
Waymo reported 96M cumulative driverless VMT through 6/30 (waymo.com/safety/impact/) and reported crossing 100M at the beginning of the day on 7/15 (www.facebook.com/102465352983...).
A run rate of ~2M miles a week - likely accelerating, but we'll assume constant for simplicity + conservatism.
7/
A run rate of ~2M miles a week - likely accelerating, but we'll assume constant for simplicity + conservatism.
7/
October 5, 2025 at 9:13 PM
Waymo reported 96M cumulative driverless VMT through 6/30 (waymo.com/safety/impact/) and reported crossing 100M at the beginning of the day on 7/15 (www.facebook.com/102465352983...).
A run rate of ~2M miles a week - likely accelerating, but we'll assume constant for simplicity + conservatism.
7/
A run rate of ~2M miles a week - likely accelerating, but we'll assume constant for simplicity + conservatism.
7/
Second, the corresponding rate for Waymo.
The author allocates 0.47 fatalities to Waymo. I'll come back to this figure later, but for now my only note is that the implied denominator of 100M VMT is too low.
6/
The author allocates 0.47 fatalities to Waymo. I'll come back to this figure later, but for now my only note is that the implied denominator of 100M VMT is too low.
6/
October 5, 2025 at 9:13 PM
Second, the corresponding rate for Waymo.
The author allocates 0.47 fatalities to Waymo. I'll come back to this figure later, but for now my only note is that the implied denominator of 100M VMT is too low.
6/
The author allocates 0.47 fatalities to Waymo. I'll come back to this figure later, but for now my only note is that the implied denominator of 100M VMT is too low.
6/
More critically, he also converts from "expected number of fatalities" (1.0) to "probability of one or more fatalities" (~63%).
For hypothesis testing the latter could be relevant, but for comparing rates, there's no reason to do this - expected fatalities (per VMT) is the more relevant figure.
5/
For hypothesis testing the latter could be relevant, but for comparing rates, there's no reason to do this - expected fatalities (per VMT) is the more relevant figure.
5/
October 5, 2025 at 9:13 PM
More critically, he also converts from "expected number of fatalities" (1.0) to "probability of one or more fatalities" (~63%).
For hypothesis testing the latter could be relevant, but for comparing rates, there's no reason to do this - expected fatalities (per VMT) is the more relevant figure.
5/
For hypothesis testing the latter could be relevant, but for comparing rates, there's no reason to do this - expected fatalities (per VMT) is the more relevant figure.
5/
Using the distribution of Waymo's driverless miles across years from Table 2 of doi.org/10.1080/1538... to match the timing, I get a blended rate of ~1.23.
The author assumes a rate of 1.0 based on pre-pandemic data, but I don't see any reason not to use more recent actuals.
4/
The author assumes a rate of 1.0 based on pre-pandemic data, but I don't see any reason not to use more recent actuals.
4/
October 5, 2025 at 9:13 PM
Using the distribution of Waymo's driverless miles across years from Table 2 of doi.org/10.1080/1538... to match the timing, I get a blended rate of ~1.23.
The author assumes a rate of 1.0 based on pre-pandemic data, but I don't see any reason not to use more recent actuals.
4/
The author assumes a rate of 1.0 based on pre-pandemic data, but I don't see any reason not to use more recent actuals.
4/
Weighting across states based on mileage from waymo.com/safety/impact/ gives 1.36 per 100M VMT blended in 2023.
2023 is the latest state-level data, but we can use national trends from crashstats.nhtsa.dot.gov/Api/Public/V... to estimate:
- 2023: 1.36
- 2024: 1.30 (-4.8%)
- 2025: 1.18 (-8.6%)
3/
2023 is the latest state-level data, but we can use national trends from crashstats.nhtsa.dot.gov/Api/Public/V... to estimate:
- 2023: 1.36
- 2024: 1.30 (-4.8%)
- 2025: 1.18 (-8.6%)
3/
October 5, 2025 at 9:13 PM
Weighting across states based on mileage from waymo.com/safety/impact/ gives 1.36 per 100M VMT blended in 2023.
2023 is the latest state-level data, but we can use national trends from crashstats.nhtsa.dot.gov/Api/Public/V... to estimate:
- 2023: 1.36
- 2024: 1.30 (-4.8%)
- 2025: 1.18 (-8.6%)
3/
2023 is the latest state-level data, but we can use national trends from crashstats.nhtsa.dot.gov/Api/Public/V... to estimate:
- 2023: 1.36
- 2024: 1.30 (-4.8%)
- 2025: 1.18 (-8.6%)
3/
First, the human driver fatality rate.
NHTSA reports fatalities per 100M vehicle miles traveled (VMT) by state and urban/rural classification: crashstats.nhtsa.dot.gov/Api/Public/V...
Rates for urban areas (where Waymo operates):
- AZ: 1.57
- CA: 1.17
- TX: 1.18
2/
NHTSA reports fatalities per 100M vehicle miles traveled (VMT) by state and urban/rural classification: crashstats.nhtsa.dot.gov/Api/Public/V...
Rates for urban areas (where Waymo operates):
- AZ: 1.57
- CA: 1.17
- TX: 1.18
2/
October 5, 2025 at 9:13 PM
First, the human driver fatality rate.
NHTSA reports fatalities per 100M vehicle miles traveled (VMT) by state and urban/rural classification: crashstats.nhtsa.dot.gov/Api/Public/V...
Rates for urban areas (where Waymo operates):
- AZ: 1.57
- CA: 1.17
- TX: 1.18
2/
NHTSA reports fatalities per 100M vehicle miles traveled (VMT) by state and urban/rural classification: crashstats.nhtsa.dot.gov/Api/Public/V...
Rates for urban areas (where Waymo operates):
- AZ: 1.57
- CA: 1.17
- TX: 1.18
2/
maybe you need to try a few at different temperatures
May 13, 2025 at 8:07 PM
maybe you need to try a few at different temperatures
I love getting Waymos with friends who’ve never tried it before, just as a reminder of how cool it is!
Most of the time it just feels like a mundane (though very good) service - it’s crazy how quickly that adjustment happened
Most of the time it just feels like a mundane (though very good) service - it’s crazy how quickly that adjustment happened
November 21, 2024 at 5:20 PM
I love getting Waymos with friends who’ve never tried it before, just as a reminder of how cool it is!
Most of the time it just feels like a mundane (though very good) service - it’s crazy how quickly that adjustment happened
Most of the time it just feels like a mundane (though very good) service - it’s crazy how quickly that adjustment happened