This week Phil & Junko talk about what it means for Autonomous Vehicle to be safe enough.
At a high level, in addition to net statistical safety, acceptable safety also requires an absence of risk hot spots. It’s not about perfection; it’s about acceptably safe:
As safe as a human driver on a net statistical basis (accounting for which car, which driver, which road, etc.)
Avoiding reckless driving
Avoiding shifting risk onto vulnerable populations
Absence of unreasonable risk (AUR)
Standards conformance
Avoiding negative externalities (e.g., not blocking fire trucks)
Accountability for breaking road rules
Meeting all stakeholder safety requirements
A single-metric approach does not work due to multiple stakeholder concerns. A multi-constraint satisfaction approach is the way to go. The bottom line: acceptable safety is meeting the threshold requirements of all your stakeholders.
This episode is a follow-on to Phil’s written piece on How Safe Is Really “Safe Enough” for Autonomous Vehicles?
You can listen to this episode on Spotify and iTunes Podcasts.
Links below point to the full episode on YouTube.
00:00 Teaser
00:59 Intro
02:33 Good enough vs. safe enough
03:14 Phil’s original definition of Safe Enough
04:36 What to you mean by: Average car
05:10 What to you mean by: Average driver
06:20 What to you mean by: Average road
08:53 Serious analysis requires a detailed comparison
10:15 Safe enough for whom?
10:31 Statistical safety is not enough
10:45 What if you are 1000 times safer than human drivers?
11:40 At less than 100 times safer, you have to ask more questions
12:08 At only 10% safer, lots of other stuff matters
12:33 Outcomes are very lagging indicators
13:40 Hypothetical example to prove net statistical safety is not enough
15:35 Another example involving tort negligence
16:41 Statistically safe is not a defense for being reckless (human or robotaxi)
17:48 The idea of risk hotspots
18:20 Risk hot spots for people
19:57 Robotaxi risk hot spots
21:40 Non-negligent robotaxi driving
23:37 Risk shifting to vulnerable populations
25:25 Women disproportionately harmed by air bags as an historical example
26:10 Driving into floods & absence of unreasonable risk (AUR), and recalls
30:00 The subtle but important difference between edible & non-toxic
30:45 Blame for mishaps is irrelevant to counting harm from crashes
32:02 Blame-free does not mean you’re safe
34:00 Don’t talk about blame; talk about whether the crash could have been avoided
35:50 Reducing risk to context, not waiting for an impending conflict
37:14 Blame is just an extreme case of poor defensive driving
38:40 Avoids all reasonably avoidable crashes & learns from experience
39:20 Would you fly on an airplane that fails to conform to industry safety standards?
41:00 Automotive does not require conforming to industry safety standards
41:40 Robotaxis don’t have a human driver to blame
42:40 Risk hot spot: fail in ways that would have been avoided with standards conformance
43:26 What we saw in San Francisco had little to do with net risk metrics
44:30 Net statistical safety does not account for other stakeholder safety requirements
45:30 The problem with a single weighted-sum metric
46:22 Discretionary road rule breaking, accountability, and human driver comparisons
48:05 All single-metrics are wrong
48:45 Satisficing solutions to optimization & squeezing fruit at the market
50:35 Safety is being acceptable (above the floor) on multiple dimensions of stakeholder constraints
51:32 The bottom line: acceptable safety is meeting the threshold requirements of all your stakeholders.
