Is the Bloom Off the Self-Driving Rose?

 

Pardon the mixed metaphor—roses don't drive—but I couldn't think of another way to summarize the current prospects for truly autonomous vehicles.  For several years now, we have been promised that self-driving cars are just around the corner.  In particular, Tesla has marketed an expensive option for their electric vehicles called "Full Self-Driving Capability."  But as the NHTSA is asking Tesla for ever more detailed information about how it has deployed and modified its autonomous-driving software since 2014, it's beginning to look like the promised future of leaving all the driving to robot chauffeurs while we nap or play cards in the back seat is nothing more than hype.

 

What is reality right now regarding self-driving cars?  The Society of Automotive Engineers has established six levels of autonomy, ranging from 0 (what a Model T had, requiring you to do everything yourself) up to Level 5.  A Level 5 car could drive you anywhere that is physically accessible by a car without your having to lift a finger. 

 

The most advanced self-driving systems currently on the market are Level 3.  A Level 3 system such as that in the Honda Legend, which was introduced only in Japan in limited quantities, really does drive the car without human intervention, but reserves the right to ask the human to take over if things get too hairy.  "Hairy" in this case can mean something as mild as a rainy day, which sets up weird reflections on streets and can confuse even deep-learning AI systems. 

 

Tesla's much-publicized "Full Self-Driving Mode" is only a Level 2 system, because the occupant is supposed to be prepared to start steering at any time.  People routinely violate this rule, however, which is how several accidents involving supposedly self-driving Teslas have happened.

 

Computer scientist and engineer Anthony Levandowsky ought to know the self-driving score if anybody does, as he got into the business way back in 2003 when he teamed with some fellow U. C. Berkeley engineers to enter a self-driving motorcycle in DARPA's 2004 Grand Challenge.  He went on to work for Google, left Google for Uber amid lawsuits charging theft of intellectual property, was convicted of same, and pardoned by President Trump on Trump's last day in office.

 

If anybody has an insider's perspective on self-driving vehicles, Levandowsky does.  What is he up to now?  He runs a company that converts giant open-pit quarry trucks to be autonomous vehicles.  And in an interview with Bloomberg News, he says that the kind of highly restricted environment in which such trucks operate may be the best that truly autonomous driving can do for the foreseeable future.

 

While there have been numerous technical advances in sensors, computing power, and AI in the two decades or so that Levandowsky has been in the business, self-driving cars are up against a truly astounding opponent:  the average driver.  As the Bloomberg article points out, suppose you see a couple of pigeons on the road ahead.  As an experienced human, you know that pigeons almost always fly away before your car lands on top of them.  And even if you are contending with a particularly dopey or hung-over pigeon whose situational awareness isn't up to snuff, running over a pigeon is not going to ruin your car.  So it's no big deal to see some pigeons in your path if you're driving.

 

But to an autonomous vehicle system that may never have encountered these particular pigeons on this particular stretch of road under these particular lighting and weather circumstances, it's a totally novel experience.  And most prudent programmers will insert a default "brake when in doubt" operation when the system encounters something that might be dangerous.  So what may well happen is that the car suddenly brakes, and the driver following you may not notice in time, leading to a rear-end collision or even a pileup on a busy freeway—all because of a pigeon.

 

Multiply this scenario by the thousands of other ones that come up all the time, and you begin to understand why self-driving taxis are found only in highly restricted areas of certain cities, and why so many of the demonstrations of self-driving cars take place in California, Arizona, and other places where clear skies can be counted on.  Levandowsky takes the position that every so-called self-driving car out there is really just a pilot project, and Tesla even makes this condition explicit, calling their system a "beta version," meaning it's still under development. 

 

So when, if ever, are we going to get to have our little cocktail among friends in the back seat, heedless of the weather or the traffic?  I can picture only two situations in which even some people can get to experience this in what remains of my lifetime.

 

One is if cities establish self-driving-only zones in which only self-driving cars with similar operating systems are allowed.  This type of operation will never be a mass-market phenomenon, but it essentially transfers Levandowsky's self-driving quarry trucks to dense downtown areas, where the environment can be controlled, perhaps with strictly-enforced rules on pedestrians.

 

The other would extend this principle to an entire region or country.  Maybe we should start with Lichtenstein.  Only self-driving cars would be allowed on the roads.  This will probably never fly in the U. S., but it might take place in some dictatorial environment such as China or North Korea—assuming North Koreans ever get rich enough to buy their own cars.  And even then, the rest of the environment is still going to cause lots of problems—pigeons can't read highway laws, and they're going to land on the road anyway. 

 

It's kind of a shame, really.  Part of me was looking forward to handing the whole responsibility of driving over to some system that had proven itself at least as trustworthy as your average taxi driver.  (Not the taxi driver I wound up with in Amherst, Massachusetts once, who had been out "fishing" with a six-pack and hit a curb, blew out a tire, stopped at a gas station, got the tire fixed, and still got us to our flight to China somehow.)  But now it looks like that's going to go the way of other Jetson-inspired dreams, like personal flying saucers.  Of course, if they do get batteries good enough to operate self-flying one-person drones, we could rewrite the FAA rules a lot more easily than redoing all the traffic laws.  But you'd still have to deal with those pesky pigeons.

 

Sources:  The Bloomberg News article "Even After $100 Billion, Self-Driving Cars Are Going Nowhere" appeared on Oct. 5, 2022 at https://www.bloomberg.com/news/features/2022-10-06/even-after-100-billion-self-driving-cars-are-going-nowhere.  I also referred to the J. D. Power website for the SAE's six levels of autonomous-vehicle operation at https://www.jdpower.com/cars/shopping-guides/levels-of-autonomous-driving-explained

and https://www.cbsnews.com/sanfrancisco/news/tesla-autopilot-driver-assist-system-nhtsa-seeks-details-recent-changes/, as well as the Wikipedia article on Anthony Levandowsky.

The Tesla Self-Driving-Software Recall

 

Innovators tend to disrupt established procedures and shake things up generally.  No matter how good a new idea is, there are usually lots of people who will be inconvenienced or worse if the innovation takes hold and spreads, and the innovator has to push hard just to get a hearing. 

 

The way Tesla under Elon Musk has introduced semi-autonomous cars is a great example of this principle.  Readers of this blog are familiar with numerous cases in which Tesla drivers have been injured or killed under circumstances that point to careless use of the car's self-driving feature.  But until now, the National Highway Traffic Safety Administration (NHTSA) has not taken definite far-reaching actions against the firm. 

 

That changed this week when Tesla, under pressure from the NHTSA, issued a voluntary recall of some 360,000 of its cars that have the so-called "Full Self-Driving" mode installed.  This option, which according to one report costs $15,000, reportedly does all the work necessary to move the car safely:  steering, acceleration, and braking, under the control of cameras and artificial-intelligence (AI) systems.  An Associated Press article quotes Raj Rajkumar, a computer-science professor at Carnegie-Mellon, as saying Teslas don't use radar or laser systems in addition to cameras, and thus can miss important environmental clues that such systems provide.

 

In its recall, the NHTSA refers to Tesla's self-driving system as a Level 2 SAE type.  Some years back the Society of Automotive Engineers established a five-level ranking system for autonomous-car features.  Warning-only features make a car Level 0, as the driver is still doing all the actual work.  Level 2 can control braking, acceleration, and steering, but the driver "must constantly supervise these support features; you must steer, brake or accelerate as needed to maintain safety." 

 

Technically, the software being recalled is a beta version, meaning that the user in some sense agrees to be a guinea pig and try out something that may still have bugs in it.  The bugs cited by the NHTSA in its recall notice include things like ignoring speed zones, rolling through stop signs without stopping, and going straight in a turn-only lane.  These things sound like typical careless-driver problems, but as Rajkumar points out, upgrading the software to solve these issues may not be a simple or quick fix.

 

The recall itself is peculiar, in that no hardware has to be idled or replaced.  Tesla will simply issue an over-the-air upgrade at no cost to the car owners.  Musk tweeted that calling such an upgrade a recall was "anachronistic and just flat wrong!"  He has a point, in that over-the-air software upgrades are a routine part of doing business, but are usually not mandated or pressured into execution by a federal agency.

 

This recall is not the only concern that the NHTSA has with self-driving Teslas.  The agency is investigating numerous incidents that pose serious safety concerns, such as the tendency of some autonomous-mode Teslas to crash into emergency vehicles—some 14 such crashes have been recorded, as well as the deaths of 19 people in accidents where self-driving features may have been involved. 

 

Musk claims that the safety record of self-driving Teslas is better than old-fashioned hand-driven cars, but detailed statistics to back up his claim are lacking.  The autonomous-car project has always suffered from a chicken-and-egg problem.  To develop good self-driving systems, you need to get extensive testing in all sorts of real-world situations, but fielding a system that isn't yet perfected—whatever that might mean—entails some level of risk for both the people riding in the autonomous vehicles and for everybody else around them too. 

 

Here in Central Texas, as I drive around the stretch of I-35 between Austin and San Antonio, sightings of Teslas have gone from remarkably rare—maybe one every few weeks—to almost routine, as a couple of them are now usually parked in the same lot I use at work.  I have not yet seen one rolling down the road while the driver was reading a book or kissing his girlfriend, but I'm sure that happens.  The one Tesla driver I've spoken to about the autonomous system—our piano tuner, of all people—says he only uses it on I-35 and takes over once he gets off the freeway.  So for every careless driver who ignores the instructions to "constantly supervise" the self-driving mode, there are many responsible Tesla owners who learn the limitations of the system and act accordingly.

 

As a federal agency, the NHTSA seems to have its act together a lot better than, say, the FBI.  It largely stays out of politics and sticks to its mandate to safeguard the nation's highways.  The Tesla recall could have been much more drastic, as the NHTSA has the power to order carmakers to tell car owners to stop using the vehicle in question until the recall is installed.  That would have been an unnecessary move.  While nineteen fatalities possibly involving a new technology are of course tragic, that number pales in comparison to the estimated 42,915 people who died in traffic accidents in 2021, which was a 16-year high. 

 

Ideally, autonomous cars will contribute to a decline in traffic casualties, not an increase.  Overall, the NHTSA seems to be doing its job as watchdog, not cutting off a given manufacturer at the knees, so to speak, but not ignoring problems either.  The recall mechanism indicates to me that the NHTSA thinks Tesla may be going a bit too fast and careless in their beta-testing of so-called Full Self-Driving systems, and Musk knows he is playing a game that could cripple his car business if he and his engineers are not careful.  But being too careful in an innovative industry leaves you in the lurch, so it will be interesting to see how, and whether, the industry as a whole approaches the prize of truly autonomous Level 5 driving, in which the driver neither knows nor cares what is going on around him.  But we are by no means there yet.

 

Sources:  The AP article "Tesla Recalls 'Full Self-Driving' To Fix Unsafe Actions" appeared on the AP website on Feb. 16, 2023 at https://apnews.com/article/tesla-recalls-full-self-driving-cars-875b54d4b71e97d43a17e968d7b856ae.  The NHTSA recall notice can be found at https://static.nhtsa.gov/odi/rcl/2023/RCLRPT-23V085-3451.PDF.  The SAE autonomous driving levels are listed at https://www.sae.org/blog/sae-j3016-update.  The statistic on 2021 U. S. driving fatalities was from https://www.cnbc.com/2022/05/17/us-traffic-deaths-hit-16-year-high-in-2021-dot-says.html. 

Driver of Tesla on Autosteer Charged with Vehicular Manslaughter

 

Shortly after the first commercially available driver-assist autopilot-equipped vehicles appeared on roadways, pundits raised the question of who would be responsible in case of an accident:  the driver, the vehicle maker, or both?  That question is about to get legs in the case of Kevin George Aziz Riad, who prosecutors have charged with two counts of vehicular manslaughter as a result of an accident that occurred on Dec. 29, 2019 in Gardena, California.

 

As we described the crash in this space shortly after it occurred, a couple in a Honda Civic were making a left turn at an intersection with the terminus of the Gardena Freeway that evening.  The traffic light at the intersection was green for them, and red for oncoming traffic to their right coming off the freeway.  Neither Riad nor the Tesla Model S he was driving paid any attention to the numerous slow-down signs or the red traffic signal when the Tesla barreled at 74 MPH into the Civic, killing both of its occupants and slightly injuring Riad and his passenger. 

 

Subsequent investigation proved that the Tesla had both Autosteer and Traffic Aware Cruise Control engaged at the time.  Tesla's instructions to drivers using these features are clear:  the driver must keep a hand on the steering wheel at all times and "be prepared to take over at any moment."  The Gardena Freeway is basically straight for at least five or six miles before its termination, and data recovered from the Tesla showed that Riad had not moved the steering wheel significantly nor applied the brakes for six minutes before the crash. 

 

Riad's defense attorney asked Los Angeles County Superior Court Judge Teresa Magno to lower the charges to misdemeanors, claiming that if a crash hadn't occurred the worst charges would have resulted from just running a red light.  The judge declined to follow that line of reasoning, and instead ruled that the trial for vehicular manslaughter will proceed.  It is widely believed that this is the first such trial involving a commercial version of automated driving technology, according to a report in the Orange County Register and a subsequent AP story.

 

Car accidents due to inattention are nothing new, but the novel feature of this case is that technology has allowed inattention to soar to new heights. 

 

In the days before driver-assist technologies such as autosteer and cruise control, one of the chief dangers of a long, straight, dull stretch of freeway was that a fatigued driver might simply fall asleep from monotony.  I'm sure this has happened to most drivers at least once or twice, and most of the time, the consequences are minor:  a slight drifting from one's lane, a jerk awake once you realize you've been dozing, and a quick flurry of attention to get things back on track.  If a fatal crash results, then the prospect of vehicular homicide charges arise, and while juries may be sympathetic to someone who simply falls asleep at the wheel, part of driving responsibly is knowing not to drive when you're very sleepy, and it's reasonable to charge such people and sometimes convict them of negligent homicide.

 

With the sophisticated driver-assist technologies of cars such as Tesla's Model S, the driver receives mixed messages.  According to an article in Popular Science, the Society of Automotive Engineers has established a five-level system for assessing how self-driving a self-driving car is, and the Model S features get it only to Level 2.  In Level 2, the driver is still in charge, even though the system can automatically brake, accelerate, and steer.  But according to Popular Science, that is not the impression that Tesla gives many of its drivers, who apparently play chicken with the system to see what they can get away with, keeping one hand on the steering wheel but having their attention otherwise engaged for many seconds or minutes, as Mr. Riad apparently did. 

 

And despite the efforts of Tesla engineers to prevent this kind of thing, they have not yet developed a psychic feature that reads the driver's mind in order to find out if he or she is really paying attention, or just acting like it with one hand on the wheel and the rest of the body doing something else altogether.  The number of fatal crashes involving Teslas with some form of driver assist engaged has reached the point that the National Highway Traffic Safety Administration (NHTSA) has required automakers to report any crashes on public roads involving such systems, including vehicles from Tesla and those of several other firms. 

 

While the absolute number of fatalities in such crashes is small, they form a leading edge of a worrisome upward trend in auto casualties generally.  After reaching a minimum in 2011, the number of U. S. auto fatalities has been creeping upward from less than 33,000 in that year to 38,824 in 2020.  A significant number of these crashes have been linked to inattentive driving in which technology—smartphones, videos, and driver-assist devices, among others—was a factor.

 

As driver-assist technology is only going to get more widespread as the cost declines and the performance increases, it's even more important that we figure out how to manage the transition between complete driver responsibility, in which nothing automated intervenes between the driver and the road and the driver recognizes that—and complete irresponsibility, the sought-for SAE Level 5 in which the car harbors an effective ideal chauffeur who allows the passengers to play pinochle, sleep, or do whatever else they want during the completely programmed ride. 

 

No Level 5 technology currently exists outside laboratories and highly-controlled environments, and it is far from clear that we will ever get there unless some radical changes are made in our entire approach to automotive transport.  In the meantime, we have to figure out a way to keep things like the Gardena crash from occurring.  Prevention may have to take the form of really annoying features such as having to press a button whenever a random light blinks, or something along those lines.  Although it would go against the grain of the Tesla-style "Look, Ma, no hands!" approach, we as a society will have to evaluate how much we want to save lives at the price of a little inconvenience.

 

Sources:  The AP story on Judge Magno's ruling to proceed to trial was carried in numerous places including ABC News at https://abcnews.go.com/Technology/wireStory/driver-stand-trial-deadly-tesla-crash-california-84851701.  I also referred to an article in the Orange County Register at https://www.ocregister.com/2022/05/19/driver-of-tesla-on-autopilot-in-fatal-gardena-crash-to-face-trial-judge-rules/, and used the sites https://www.popsci.com/technology/autonomous-vehicles-explained/

and https://www.kktv.com/2022/05/18/federal-agency-sends-team-probe-tesla-crash-that-killed-3/as well.  My first blog on this incident is at https://engineeringethicsblog.blogspot.com/2020/01/are-self-driving-cars-more-dangerous.html.

The Fatal Tesla Crash in Houston: What Really Happened?

 

Late on Saturday, Apr. 17, Dr. Will Varner, an anesthesiologist, said good-by to his wife and the wife of a friend, Everett Talbot, as the women left Varner's Tesla Model S for a home in a gated community in The Woodlands outside of Houston, Texas.  Apparently, Varner then offered to show Talbot "how it can drive itself" on a short test drive around the neighborhood, as subsequent testimony from witnesses indicates.  What is certain is that around 11:25 PM, the Tesla went off the road and crashed at considerable speed into a tree.  The car's massive lithium-ion battery caught fire, and photographs of the wreckage after the bodies were removed show only two door uprights standing on either side of the otherwise flattened and blackened wreckage.  First responders found the body of Talbot in the front passenger seat and that of Varner in the back seat.  Neither was at the wheel of the vehicle at the time of the crash.

 

Harris County Constable Mark Herman claimed to reporters that "no one was driving" the 2019 Tesla at the time of the crash.  But in a tweet the following Monday, Tesla CEO Elon Musk stated "Data logs recovered so far show Autopilot was not enabled and this car did not purchase FSD."  FSD stands for Full Self-Driving, a mode which still requires driver supervision.  Musk went on to say "Moreover, standard Autopilot would require lane lines to turn on, which this street did not have."

 

In defense of Tesla and its CEO, Tesla drivers are warned repeatedly to keep their hands on the wheel even if Full Self Driving mode is engaged.  This is like telling a five-year-old to keep your hand in the cookie jar, but just don't take any cookies.  Many Tesla drivers have given in to the temptation to engage Autopilot or otherwise surrender control of the vehicle to the system computer and allow their attention to stray, or even leave the driver's seat altogether, as Dr. Varner apparently did.  And the self-driving capabilities of the car are good enough that most of the time, absentee drivers can get away with it.

 

Musk bases his claim that Autopilot was not enabled on the fact that Teslas telemeter "periodic" updates via wireless links to the company.  Leaving aside the question of whether having your car inform Tesla of your every driving move is compatible with privacy, it is not clear how frequent these updates are.  If you read Musk's tweet like a lawyer, the phrase "Data logs recovered so far" could cover the possibility that the most recent data log Tesla has from the vehicle in question was many minutes before the actual crash occurred.  In other words, Musk could be saying nothing more significant than, "We know that ten minutes before the crash happened, Autopilot was not engaged."  But a lot can happen in ten minutes. 

 

The Houston police authorities have both impounded the wreckage of the Tesla and stated that they "eagerly wait" for the data that Tesla has recovered remotely.  It is unclear at this writing whether any data logs can be recovered from the incinerated wreck.  Unless Tesla has taken steps to harden the housing of the car's computer memory in a way similar to the kind of waterproofing and fireproofing that aviation black boxes have, I'd say that the remote data is all the data that anyone's likely to recover.

 

In the long run, removing the human element entirely from driving may significantly reduce automotive fatalities and injuries.  And when so-called "driver assist" systems such as lane-keeping, station-keeping at a fixed distance behind a leading vehicle, and automatic braking in emergencies are employed in the way they are supposed to be used—as assists to a real driver at the wheel, not as a substitute—studies have shown that they do reduce accidents. 

 

But the way Tesla has marketed their vehicles and promoted the driver-assist features as "Full Self-Driving" and "Autopilot" is misleading on the face of it.  Musk's cowboy reputation, which he appears to relish, may be a big reason why only 14% of Americans say they "would trust riding in a vehicle that drives itself."  If Musk really intends to sell a mass-market car rather than one that only doctors and stockbrokers can afford, that 14% number will have to increase a lot before a truly self-driving car can succeed.

 

In the meantime, deceptive and hypocritical marketing such as Tesla engages in contributes to the perception that alone among automakers, Tesla has really arrived at what the Society of Automotive Engineers calls "level 5" autonomous driving.  In a letter to California's Department of Motor Vehicles in March of this year, Tesla representatives admitted that the most advanced features of any Tesla vehicle amount only to Level 2 autonomy.  According to the SAE, Level 2 automation is simply driver-support, not automated driving, and requires that the driver constantly supervise the car's operation.  Clearly, many Tesla drivers are going beyond that.  Dr. Varner gambled on getting away with it and lost.

 

There's nothing new about automakers providing features on cars that some drivers abuse.  The muscle cars of the 1960s had power and acceleration that went way beyond anything normal driving required, and as a consequence some people wound up dying in fiery crashes after 140-mph joyrides.  But bigger engines and faster cars were just incremental changes that took place since the invention of the automobile.

 

Cars that seem to drive themselves are something truly new in automotive history, and we are still in the early stages of seeing how autonomous driving will play out.  While Tesla deserves credit for marketing what is probably the most technically advanced combination of driver-assist technologies on the market today, they have created a dangerous situation in which even a few spectacular crashes such as the one in Houston can put a damper on an entire technology and scare people away from it. 

 

If Tesla is smart enough to make a nearly autonomous car, they are also smart enough to figure out how to keep drivers from absenting themselves from the wheel until the future date when it is reasonably safe to do so.  By Tesla's own admission, that date is not here yet, but it is apparently ridiculously simple to fool a Tesla car into driving itself while you play your violin in the back seat.  Other car makers are taking more sophisticated precautions such as eye-motion detection to ensure that the driver-assist system always has a driver to assist who is paying attention.  It's way past time for Tesla to do something like the same.

 

Sources:  I consulted the following articles in the preparation of this blog:  "Feds investigating fatal Tesla crash in Texas" (Austin American-Statesman, 4/20/2021), "Tesla crash shows Autopilot isn't there yet," (Austin American-Statesman, 4/22/2021), Car and Driver, https://www.caranddriver.com/news/a36189237/tesla-model-s-fire-texas-crash-details-fire-chief/,

Reuters at https://www.reuters.com/business/autos-transportation/us-probes-fatal-tesla-crash-believed-be-driverless-2021-04-19/, the New York Post at https://nypost.com/2021/04/21/victims-in-deadly-houston-telsa-crash-identified/, Click2Houston.com at https://www.click2houston.com/news/local/2021/04/18/2-men-dead-after-fiery-tesla-crash-in-spring-officials-say/, and the Society of Automotive Engineers at https://www.sae.org/news/press-room/2018/12/sae-international-releases-updated-visual-chart-for-its-%E2%80%9Clevels-of-driving-automation%E2%80%9D-standard-for-self-driving-vehicles. 

Are Self-Driving Cars More Dangerous?

Around midnight Sunday, Dec. 29, 2019, the driver of a Honda Civic headed northbound on Vermont Avenue in Gardena, California was making a left turn from Vermont onto Artesia Boulevard.  The traffic light at the intersection was red for westbound traffic on Artesia, and the intersection happened to be the western end of the Gardena Freeway, where it becomes a surface road.  At that moment, a Tesla Model S zoomed off the freeway westbound through the red light and crashed into the Honda, killing its two occupants.  The Tesla driver and his passenger were not seriously injured.  Early news reports failed to indicate whether the Tesla was on autopilot, but National Highway Traffic Safety Administration (NHTSA) officials are investigating the crash to determine whether the autopilot was engaged.

This latest fatality involving an autopilot-equipped Tesla inspired an Associated Press review of recent fatalities involving Tesla cars in which the autopilot was engaged.  The curious reader can view the website www.tesladeaths.com, where someone has attempted to compile a complete list of worldwide statistics for fatal crashes involving Tesla cars.  As of the end of 2019, the list totaled 110 deaths, of which only 4 are in the category of "verified Tesla autopilot death."  As well over 200,000 Teslas have been sold, these statistics are not particularly remarkable, except for the fact that Tesla purports to be the leading edge of the automotive future.  As such, it deserves closer scrutiny, and that is what it's getting.

The problem in answering the question of our headline is, "more dangerous than what?"  Not only is Tesla the world's best-selling plug-in passenger car, it offers what many regard as the most sophisticated commercially available autopilot system as well.  And in contrast to the more conservative approach many automakers have taken in adding self-driving features such as lane following and automatic braking, the Tesla driver can turn on autopilot and let go of the wheel.  Such behavior is not advised by Tesla, but since when have instruction manuals been 100% effective in keeping people from doing stupid things?

The Associated Press article quotes Jason Levine, head of the nonprofit Center for Auto Safety in Washington, as saying, “At some point, the question becomes: How much evidence is needed to determine that the way this technology is being used is unsafe?”  Levine criticized the NHTSA for dragging its feet instead of issuing regulations as to how Tesla's autopilot feature can be used.  Simply warning the driver that he or she should be alert at all times when the autopilot is working doesn't make it happen.  At least two fatal U. S. crashes (one in Florida and one in Ohio) happened when the autopilot's sensors became confused and failed to recognize a large truck blocking the roadway.  Presumably, if the drivers had been paying attention, they might have seen the truck and stopped.

Promoters of the autonomous-vehicle future face two distinct but interrelated obstacles that could delay or even prevent widespread adoption of self-driving cars.

The first obstacle is technology.  The Society of Automotive Engineers (SAE) has defined five levels of automated driving.  Level 0 is a 1955 Plymouth—completely manual operation—and Level 5 would be the equivalent of an electronic chauffeur—the passenger can watch TV, sleep, or do anything else you would do if you knew a trusted and competent human driver was in charge.  No automaker currently offers a Level 5 vehicle for sale, but Elon Musk is claiming that early this year, Tesla will start to sell fully self-driving cars, which sounds like Level 5 to me.  Of course, this may be nothing but vaporware.  But unless some pretty radical improvements are made in autopilot technology, it's inevitable that some fatal crashes will happen in which the autopilot was engaged.

And here's where we run into the second obstacle:  public perception, including the perception of government regulators, lawmakers, and their constituents.  I don't know about you, but I would feel a lot worse thinking about dying in a car wreck in which my car was driving itself, rather than dying in one where I was actively at the wheel.  True, I'd be just as dead in either case, but there's something about the hope that one can make a difference if one is trying to control the situation.  This is not a completely rational state of mind, but carmakers learned decades ago to appeal to the sub-rational "lizard brain" of the consumer.  Why else do pickup ads show their products bounding over rugged mountains and doing extreme feats that 99% of drivers will never have to do? 

The budding autonomous-car industry is still treading on very shaky ground, at least in the U. S., where the majority of fatal accidents involving Teslas have occurred.  As the statistics show, less than 10% of fatal crashes involving Teslas are associated with the use of the autopilot.  But statistics do not count for much in public perception, and Elon Musk's cowboy-style reputation lends credibility to the accusation that he and his company are playing games with the safety of their customers, and by implication, with the safety of anyone within collision range of a Tesla.

If properly designed and deployed, I concede that autonomous vehicles could lower the rate of traffic fatalities while lessening traffic congestion and doing other good things such as reducing carbon emissions as well.  But there is a world of challenges in that "if."  There may be unknown factors that no one will discover until there is a certain critical mass of autonomous vehicles on the road already.

In statistical mechanics involving solutions of, say, salt in water, you can apply simple rules to very dilute solutions, so dilute that each salt molecule or atom can be treated like it is the only one in the solution.  So far, autonomous vehicles are so rare that each one is surrounded by a sea of non-autonomous vehicles, and the software probably operates under that assumption.

But when you put enough salt in the water so that each salt atom gets within shouting distance of another salt atom, things get complicated.  New effects such as saturation and crystallization occur, and your analysis has to be more sophisticated in order to deal with these effects. 

If autonomous vehicles, especially those made by different manufacturers, ever become common enough so that one vehicle can "see" another one in a typical driving situation, it is very likely that novel and perhaps hazardous effects will occur that even the designers may not have anticipated.  But that will never happen if the public gets so fearful of accidents involving autopiloted cars that they are regulated out of existence.  I hope that doesn't happen either, but if Musk and Tesla get too careless, they might end up triggering just such a reaction.

Sources:  The Associated Press article I referred to appeared in many locations, among which was the San Jose Mercury-News website at https://www.mercurynews.com/2020/01/03/3-crashes-3-deaths-raise-questions-about-teslas-autopilot/.  I also referred to the same site for information on the Gardena crash at

 https://www.mercurynews.com/2020/01/02/fatal-tesla-crash-in-california-investigated-by-feds/.  The Tesla fatal-crash statistics website is www.tesladeaths.com, and I also referred to the Wikipedia article on Tesla, Inc.