Remote Operators and Their Accountability
Driverless cars do not mean human-less driving. Human operators contribute a lot to autonomous driving. Our inaugural video podcast focuses on a detailed look into their roles and accountability.
By Junko Yoshida
If you haven’t yet tuned into the inaugural “Phil & Junko on AV Safety” podcast, you’re in for a treat.
I’ve known Phil Koopman, safety expert and emeritus professor at Carnegie Mellon, for many years. I’ve always appreciated his help when my reporting needed to put Autonomous Vehicle (AV) safety matters into perspective.
But it was during this conversation about safety engineering—a topic Phil has been talking about all these years—that its core truth hit home with me: The heart and soul of safety engineering is “identifying hazards and mitigating them.” Equally important, although often overlooked, is the job of clarifying the roles, responsibility and accountability performed by machines, humans and anything else involved in system operations.
Our first podcast episode centers on human remote assistance operators.
Forget the overly simplistic argument that robocar promoters use to fit the roles of human remote assistance operators into two bins: “drivers” or “not drivers.”
Phil breaks down, meticulously, the many different roles played by humans, either in-cabin or remotely, into twelve categories. For each, he gives us an example of the challenges and responsibilities that human job entails.
Why, you might ask, would Phil go to such an extent in dissecting remote assistance operations?
First and foremost, Phil is an engineer. He can’t help but be methodical and fastidious about every little thing.
Of course, journalists like me would prefer the much simpler black-and-white narrative pushed by some robotaxi companies, and perpetuated by some reporters and even auto industry pundits. If remote operators are simply dispensing guidance to AVs, all good. It’s like having your AV “phone a friend” (a la Waymo). But when remote operators actually start to drive an AV remotely (a la Tesla, and reportedly some Chinese robotaxis), that’s where the rubber departs the road.
False dichotomy
Phil calls the “driver” vs “non-driver” argument “a false dichotomy.” When you examine roles played by each human operator, the necessary tasks are more nuanced. “As it turns out, everyone’s a little bit of a driver no matter which role they play,” according to Phil.
He reminds us that there is nothing wrong with human operators offering a “decision or approval” to questions posed by AVs. “What’s happening is the car’s computer is saying, I really want human judgment to weigh in here,” he explained.
As AV technology stands today, “I can’t imagine putting these vehicles on the road and not having that [remote operation] capability.”
However, what’s eating safety experts like Phil is an unfortunate trend in the AV industry to insist that remote assistance operators who don’t drive have no safety role. They say that it is the responsibility of the autonomous vehicle itself which makes the final decision.
But the hitch is that any advice given to AVs by human assistance operators affects the vehicle’s movements. God forbid that said movement ends up on top of a little kid and his bicycle. In this light, isn’t it self-evident that human remote operators are doing tasks that have something to do with safety?
In our society, people who make honest mistakes are nonetheless accountable. As a simple example, if your kid mistakenly throws a ball into a neighbor's window and breaks it, that does not mean anyone is going to jail. But your neighbor can reasonably expect your kid (or you) to pay for the repair.
Still, AV proponents embrace cognitive dissonance and insist on placing all accountability for a driving mistake on the computer driver, instead of human remote operators. How come?
Accountability blame game
Sorting through this blame game is where Phil insights are most illuminating.
The AV industry says human remote operators have nothing to do with safety. But when asked about these call-center staffers’ qualifications, their bosses dwell incongruously on their safety record and how closely they are monitored.
Phil wonders why that is.
Answering his own question, he said, “The answer is that remote operators do have something to do with safety. AV companies just don’t want to admit it.”
He speculates:
“Once you acknowledge the safety roles of remote operators, that brings monitoring, accountability, plus regulatory and enforcement mechanisms into play.”
While “Phil & Junko on AV Safety” isn’t a legal podcast, Phil doesn’t hesitate to broach liability issues. When something goes wrong with AVs, the accountability blame game can create a convenient void in apparent responsibility, making it difficult for plaintiffs seeking compensation for injuries or damage.
Plaintiffs will struggle to cite mistakes made by remote operators, because the AV industry says they have nothing to do with safety.
The only option left is to sue the “computer driver” as a product liability issue, arguing that the computer made a mistake. “But the catch is that the rules of engagement for product liability are much different” from tort negligence, pointed out Phi. “It’s much more expensive, it’s much harder to win.”
Plaintiffs may well end up with a case in which a remote assistant has no safety roles to play, according to the claims by the AV industry, evading tort negligence. Meanwhile, the AV might not have any design flaws. Or, because it’s hard to prove, the legal team might never find anything wrong with the computer driver’s design that can be blamed. There might be a path through this legal mess, but it will be neither quick nor cheap to navigate it.
In a nutshell, this could leave victims of an AV crash hanging. It seems likely to get harder for plaintiffs to get compensated if an autonomous vehicle causes harm.
Senator Markey’s investigation
As I noted in the podcast, I’m really glad that we’ve come to the point that the issue of human remote assistance operators has become a topic of investigation, led by Senator Ed Markey (D-Mass).
Robotaxi companies, once reluctant to acknowledge the existence of human remote operators and their reliance on them, are now compelled to walk back their narrative.
While they stick to their claim that autonomous vehicles are smart and safe enough to make final driving decisions autonomously, the cat’s out of the bag: Driverless does not mean humanless.
In our podcast, I asked Phil whether Sen. Markey asked AV companies the right questions. Did he illuminate the real issues of remote operations? What did he miss? Where does his investigation need to focus?
The answer comes back to the accountability of remote operators, which Senator Markey didn’t exactly highlight.
You’ll find out more on this topic, and what we are thinking about the responsibility and accountability of human remote assistance operators in this podcast.
Please sign up for our free “Phil & Junko on AV Safety” stories and newsletter on Substack. Make sure to watch the full Ep. 1 of our video podcast on YouTube.
This post is part of a collaboration between Phil Koopman & Junko Yoshida on AV Safety. Please subscribe for free if you’d like to receive future posts.
