IEEE Technology and Society Magazine - March 2018 - 64

The safety of AVs may become a
scientific proxy for an economic
conflict between professional drivers
and robotics manufacturers who are
trying to replace them.

Crash Rate per Million
Miles Driven

of 1.09 fatalities per 100 million [vehicle] miles [trav-
eled]." Precise estimates of such rare events require
large samples. In the case of human-driven vehicles,
these large samples come from trillions of vehicle
miles traveled per year, distributed over tens of mil-
lions of cars.
Kalra and Paddock [7] calculate the number of AV
miles traveled required to produce statistically precise
estimates of AV crash rates across several different
crash types (from any reported crash to fatalities),
degrees of statistical stringency or precision, and statis-
tical tasks (estimating the maximum crash rate, demon-
strating that the crash rate is lower than a threshold,
and demonstrating that the crash rate is statistically sig-
nificantly lower than the human crash rate). Their calcu-
lations range across 5 orders of magnitude, from 1.6 million

25
20
15
10
5
0

Level 1

Level 2
Crash Severity

Level 3

SHRP 2 Overall
SHRP 2 PR
Self-Driving Car
SHRP 2 Overall Age-Adjusted
SHRP 2 PR Age-Adjusted
Figure 1. Crash rate estimates for human and AV system
drivers. Bars give crash rate estimates with whiskers indicating
95% confidence intervals. Blue and orange bars are different
classes of estimates for human drivers; green bars are
estimates for AVs. Level 1 crash severity is the most severe
and level 3 is the least severe. Note that, for levels 1 and 2, the
confidence intervals for humans are entirely contained within
the confidence intervals for AVs. This is due to limited data for
AVs [8, Fig. 3, 23].

64

miles (95% confidence of a maximum crash rate of
190 reported crashes per 100 million miles traveled) to
11 billion miles (95% confidence and 80% power to
detect a 20% improvement over the human fatality
rate of 1.09 fatalities per 100 million miles). A fleet of
1000 AVs, driving an average of 6 hours per day at
an average of 60 mph, would require 84 years to travel
11 billion miles.
Uncertainty due to the large amount of data required
is reflected in other analyses. A Google-sponsored
report used data from a National Academies study and
Google to compare human-driven and automated vehi-
cle crash rates [8]. The 95% confidence intervals for the
AV crash rate estimates are several times wider than
those for human-driven vehicles, and for the two high-
est-severity crash levels, the AV confidence intervals
fully contain the human-driven confidence intervals
[8, fig. 3, p. 23]. There is simply insufficient observation-
al evidence to conclude that AVs are statistically signifi-
cantly safer than human-driven cars at conventional
levels of statistical stringency. And, per [7], this evidence
will remain insufficient for decades to come.

Alternatives to Observational Data
If we can't use observational data, then simulations,
physical test systems, and other broadly "laboratory-
based" or "experimental" methods seem to offer a way
forward. For example, self-driving software might be run
offline several orders of magnitude faster than real-time
driving, and thereby generate billions of "virtual miles
traveled" on a scale of years - or perhaps even days  -
rather than decades.
An announcement by Tesla in September 2016 sug-
gests another testing strategy. In May 2016, a Tesla driv-
er was killed when his car when it was operating in
"Autopilot" mode and failed to recognize a truck. In
response, Tesla proposed a "fleet learning" strategy: A
Tesla car's AV system will track its human drivers'
behavior while the AV system is "off," comparing the
human behavior to the behavior that it would take
based on its sensor data. Pooling these data from all
Tesla vehicles will enable the identification of locations
and situations where the AV system tends to be inaccu-
rate and needs improvement [9].
This fleet learning proposal could also be used for
safety studies. For example, fleet learning data could
be filtered for three kinds of scenarios: a crash
occurred under AV system control; a crash occurred
under human control; and a human reclaimed control
from the AV system and immediately began rapidly
braking or changing direction. The first kind of scenario
gives us direct data on AV crash rates. The second and
third scenarios let us compare human behavior with
counterfactual AV system behavior. In the third scenario,

IEEE Technology and Society Magazine

∕

march 2018



Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2018

Contents
IEEE Technology and Society Magazine - March 2018 - Cover1
IEEE Technology and Society Magazine - March 2018 - Cover2
IEEE Technology and Society Magazine - March 2018 - 1
IEEE Technology and Society Magazine - March 2018 - Contents
IEEE Technology and Society Magazine - March 2018 - 3
IEEE Technology and Society Magazine - March 2018 - 4
IEEE Technology and Society Magazine - March 2018 - 5
IEEE Technology and Society Magazine - March 2018 - 6
IEEE Technology and Society Magazine - March 2018 - 7
IEEE Technology and Society Magazine - March 2018 - 8
IEEE Technology and Society Magazine - March 2018 - 9
IEEE Technology and Society Magazine - March 2018 - 10
IEEE Technology and Society Magazine - March 2018 - 11
IEEE Technology and Society Magazine - March 2018 - 12
IEEE Technology and Society Magazine - March 2018 - 13
IEEE Technology and Society Magazine - March 2018 - 14
IEEE Technology and Society Magazine - March 2018 - 15
IEEE Technology and Society Magazine - March 2018 - 16
IEEE Technology and Society Magazine - March 2018 - 17
IEEE Technology and Society Magazine - March 2018 - 18
IEEE Technology and Society Magazine - March 2018 - 19
IEEE Technology and Society Magazine - March 2018 - 20
IEEE Technology and Society Magazine - March 2018 - 21
IEEE Technology and Society Magazine - March 2018 - 22
IEEE Technology and Society Magazine - March 2018 - 23
IEEE Technology and Society Magazine - March 2018 - 24
IEEE Technology and Society Magazine - March 2018 - 25
IEEE Technology and Society Magazine - March 2018 - 26
IEEE Technology and Society Magazine - March 2018 - 27
IEEE Technology and Society Magazine - March 2018 - 28
IEEE Technology and Society Magazine - March 2018 - 29
IEEE Technology and Society Magazine - March 2018 - 30
IEEE Technology and Society Magazine - March 2018 - 31
IEEE Technology and Society Magazine - March 2018 - 32
IEEE Technology and Society Magazine - March 2018 - 33
IEEE Technology and Society Magazine - March 2018 - 34
IEEE Technology and Society Magazine - March 2018 - 35
IEEE Technology and Society Magazine - March 2018 - 36
IEEE Technology and Society Magazine - March 2018 - 37
IEEE Technology and Society Magazine - March 2018 - 38
IEEE Technology and Society Magazine - March 2018 - 39
IEEE Technology and Society Magazine - March 2018 - 40
IEEE Technology and Society Magazine - March 2018 - 41
IEEE Technology and Society Magazine - March 2018 - 42
IEEE Technology and Society Magazine - March 2018 - 43
IEEE Technology and Society Magazine - March 2018 - 44
IEEE Technology and Society Magazine - March 2018 - 45
IEEE Technology and Society Magazine - March 2018 - 46
IEEE Technology and Society Magazine - March 2018 - 47
IEEE Technology and Society Magazine - March 2018 - 48
IEEE Technology and Society Magazine - March 2018 - 49
IEEE Technology and Society Magazine - March 2018 - 50
IEEE Technology and Society Magazine - March 2018 - 51
IEEE Technology and Society Magazine - March 2018 - 52
IEEE Technology and Society Magazine - March 2018 - 53
IEEE Technology and Society Magazine - March 2018 - 54
IEEE Technology and Society Magazine - March 2018 - 55
IEEE Technology and Society Magazine - March 2018 - 56
IEEE Technology and Society Magazine - March 2018 - 57
IEEE Technology and Society Magazine - March 2018 - 58
IEEE Technology and Society Magazine - March 2018 - 59
IEEE Technology and Society Magazine - March 2018 - 60
IEEE Technology and Society Magazine - March 2018 - 61
IEEE Technology and Society Magazine - March 2018 - 62
IEEE Technology and Society Magazine - March 2018 - 63
IEEE Technology and Society Magazine - March 2018 - 64
IEEE Technology and Society Magazine - March 2018 - 65
IEEE Technology and Society Magazine - March 2018 - 66
IEEE Technology and Society Magazine - March 2018 - 67
IEEE Technology and Society Magazine - March 2018 - 68
IEEE Technology and Society Magazine - March 2018 - 69
IEEE Technology and Society Magazine - March 2018 - 70
IEEE Technology and Society Magazine - March 2018 - 71
IEEE Technology and Society Magazine - March 2018 - 72
IEEE Technology and Society Magazine - March 2018 - 73
IEEE Technology and Society Magazine - March 2018 - 74
IEEE Technology and Society Magazine - March 2018 - 75
IEEE Technology and Society Magazine - March 2018 - 76
IEEE Technology and Society Magazine - March 2018 - 77
IEEE Technology and Society Magazine - March 2018 - 78
IEEE Technology and Society Magazine - March 2018 - 79
IEEE Technology and Society Magazine - March 2018 - 80
IEEE Technology and Society Magazine - March 2018 - 81
IEEE Technology and Society Magazine - March 2018 - 82
IEEE Technology and Society Magazine - March 2018 - 83
IEEE Technology and Society Magazine - March 2018 - 84
IEEE Technology and Society Magazine - March 2018 - 85
IEEE Technology and Society Magazine - March 2018 - 86
IEEE Technology and Society Magazine - March 2018 - 87
IEEE Technology and Society Magazine - March 2018 - 88
IEEE Technology and Society Magazine - March 2018 - 89
IEEE Technology and Society Magazine - March 2018 - 90
IEEE Technology and Society Magazine - March 2018 - 91
IEEE Technology and Society Magazine - March 2018 - 92
IEEE Technology and Society Magazine - March 2018 - Cover3
IEEE Technology and Society Magazine - March 2018 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com