Federal Safety Agency Investigates Tesla Autopilot

 

In 2015, the upstart automaker Tesla introduced its Autopilot feature, an advanced artificial-intelligence-enabled system that takes over most of the routine operations that a human driver normally performs.  At the same time, Tesla warned Autopilot users that they should remain attentive with their hands on the wheel at all times, even when Autopilot is engaged.

 

This is a little bit like taking a hungry child into a candy store and telling them not to touch anything.  Most kids will obey, but it's hard on the kid and it can lead to embarrassing situations.

 

It's not surprising that, according to the U. S. National Highway Traffic Safety Administration (NHTSA), since 2016 ten people have died in eight crashes of Tesla vehicles in which the Autopilot feature was the cause of the crash.  Lately, there have been numerous crashes, one of them fatal, in which Teslas with Autopilot engaged have run into the rear of emergency vehicles with flashing lights. 

 

Finally, the NHTSA has had enough.  It has launched a formal investigation into how the Autopilot system works, how it is implemented, what its defects are, and what steps Tesla has taken to make sure that drivers are paying attention like they are supposed to when the Autopilot is driving the car.  There is abundant evidence that in many of the crashes, the driver was doing something other than watching the road:  watching a movie, playing a video game, or even sitting in a seat other than the driver's seat.  The Autopilot system is supposed to monitor hand pressure on the steering wheel, but according to some sources, this feature is very easy to defeat, and many people appear to have done so.  And most of them probably get away with it most of the time.  But not always.

 

In human-machine safety issues, there is a tradeoff between the two poles which represent extreme approaches to operating a device safely.  One pole relies totally on training the individual not to do dangerous things, or to do them in a safe way, if that makes sense.  Think of stunt drivers in the movie business:  they do things with cars that cars are not designed to do, but with careful planning and finely-honed skills, they manage to survive car flips, crashes, and other tricks that have probably passed into history now that CGI technology is so good that real stunt drivers are probably looking for work.  But the point is that this approach to safety concentrates on the knowledge and attention of the operator or driver, and basically tells him or her to drive safely.

 

The other pole of safety is building in foolproof safety features to the machine itself, so that even an ignorant five-year-old turned loose with the keys couldn't get hurt.  It's not possible to make a car at a reasonable price that is completely safe no matter what you do—at least not yet.  But many of the autonomous-vehicle-type features that are now showing up on many makes besides Tesla move in this direction:  lane-keeping features, automatic braking to avoid head-on collisions, and so on.  They make up for a driver's deficiencies, inattention, or errors.  But they are far from perfect yet, and so the attention and intelligence of the driver are still needed to fill in the gaps where systems like Tesla's Autopilot still can't figure out the situation, such as an emergency vehicle stopped in your lane.

 

I expect the NHTSA will encounter some headwinds in trying to figure out Tesla's Autopilot system.  Elon Musk has, shall we say, a rather cavalier attitude toward convention and traditional ways of doing things, and recently abolished Tesla's public-relations department.  Perhaps he thinks a few tweets from him do just as well, and in the absence of more formal ways of getting information from the company, he may be right.  But nobody can stop the NHTSA from renting or buying some Teslas and putting them through various scenarios and seeing what they do with and without human supervision.  Whatever is going on under the hood, the results will be clear to see.

 

But just testing the hardware and software is only part of the issue.  The poisonous mixture that the NHTSA is dealing with combines an Autopilot system that is very good—so good that people really can let it drive the car for many minutes at a time and get away with it—and drivers who either intentionally put too much trust in the Autopilot system, or simply get distracted and fail to do what they know they ought to be doing, which is looking at the road.  But nobody just accidentally starts watching a movie or playing a video game, and so we must conclude that in at least some of the cases where inattention and the Autopilot have caused crashes, people simply ignored the advice of Tesla to not let Autopilot drive the car by itself, and paid the penalty for their inattention.

 

Now in some countries and cultures (and political persuasions—notably extreme libertarianism), this would not be a concern of the government's.  If people want to do foolish things and ignore instructions, well, let them do it and suffer the consequences.  The problem with this attitude is that it ignores everybody else, particularly other people who might be harmed and killed in the same accident.

 

My point is simply that we in the U. S. have grown accustomed to holding automakers to safety standards that avoid preventable accidents, in the sense that preventable accidents follow a consistent pattern which reasonable interventions at not too much cost can prevent. 

 

We are in a curious transition phase in which systems like Autopilot are good enough to fool us that they can really drive our cars without us paying any attention, but not good enough to do it for real.  And until it is just as safe to play pinochle from the driver's seat as it is at home, we need some way to remind drivers that they can't ignore the road even if the car seems to be driving itself.

 

Sources:  I consulted an article in Consumer Reports at https://www.consumerreports.org/autonomous-driving/nhtsa-safety-defect-investigation-tesla-autopilot-crashes-a6996819019/ and an article in the Aug. 16 Austin American-Statesman online edition "Feds Open Investigation Into Tesla's Autopilot System.  The statistic on total fatalities and accidents due to Autopilot since 2016 was obtained from https://thehill.com/changing-america/sustainability/infrastructure/561717-increasing-number-of-crashes-involving-teslas#:~:text=In%20total%2C%20at%20least%2010,each%20year%20in%20the%20U.S.

The Fatal Tesla Crash in Houston: What Really Happened?

 

Late on Saturday, Apr. 17, Dr. Will Varner, an anesthesiologist, said good-by to his wife and the wife of a friend, Everett Talbot, as the women left Varner's Tesla Model S for a home in a gated community in The Woodlands outside of Houston, Texas.  Apparently, Varner then offered to show Talbot "how it can drive itself" on a short test drive around the neighborhood, as subsequent testimony from witnesses indicates.  What is certain is that around 11:25 PM, the Tesla went off the road and crashed at considerable speed into a tree.  The car's massive lithium-ion battery caught fire, and photographs of the wreckage after the bodies were removed show only two door uprights standing on either side of the otherwise flattened and blackened wreckage.  First responders found the body of Talbot in the front passenger seat and that of Varner in the back seat.  Neither was at the wheel of the vehicle at the time of the crash.

 

Harris County Constable Mark Herman claimed to reporters that "no one was driving" the 2019 Tesla at the time of the crash.  But in a tweet the following Monday, Tesla CEO Elon Musk stated "Data logs recovered so far show Autopilot was not enabled and this car did not purchase FSD."  FSD stands for Full Self-Driving, a mode which still requires driver supervision.  Musk went on to say "Moreover, standard Autopilot would require lane lines to turn on, which this street did not have."

 

In defense of Tesla and its CEO, Tesla drivers are warned repeatedly to keep their hands on the wheel even if Full Self Driving mode is engaged.  This is like telling a five-year-old to keep your hand in the cookie jar, but just don't take any cookies.  Many Tesla drivers have given in to the temptation to engage Autopilot or otherwise surrender control of the vehicle to the system computer and allow their attention to stray, or even leave the driver's seat altogether, as Dr. Varner apparently did.  And the self-driving capabilities of the car are good enough that most of the time, absentee drivers can get away with it.

 

Musk bases his claim that Autopilot was not enabled on the fact that Teslas telemeter "periodic" updates via wireless links to the company.  Leaving aside the question of whether having your car inform Tesla of your every driving move is compatible with privacy, it is not clear how frequent these updates are.  If you read Musk's tweet like a lawyer, the phrase "Data logs recovered so far" could cover the possibility that the most recent data log Tesla has from the vehicle in question was many minutes before the actual crash occurred.  In other words, Musk could be saying nothing more significant than, "We know that ten minutes before the crash happened, Autopilot was not engaged."  But a lot can happen in ten minutes. 

 

The Houston police authorities have both impounded the wreckage of the Tesla and stated that they "eagerly wait" for the data that Tesla has recovered remotely.  It is unclear at this writing whether any data logs can be recovered from the incinerated wreck.  Unless Tesla has taken steps to harden the housing of the car's computer memory in a way similar to the kind of waterproofing and fireproofing that aviation black boxes have, I'd say that the remote data is all the data that anyone's likely to recover.

 

In the long run, removing the human element entirely from driving may significantly reduce automotive fatalities and injuries.  And when so-called "driver assist" systems such as lane-keeping, station-keeping at a fixed distance behind a leading vehicle, and automatic braking in emergencies are employed in the way they are supposed to be used—as assists to a real driver at the wheel, not as a substitute—studies have shown that they do reduce accidents. 

 

But the way Tesla has marketed their vehicles and promoted the driver-assist features as "Full Self-Driving" and "Autopilot" is misleading on the face of it.  Musk's cowboy reputation, which he appears to relish, may be a big reason why only 14% of Americans say they "would trust riding in a vehicle that drives itself."  If Musk really intends to sell a mass-market car rather than one that only doctors and stockbrokers can afford, that 14% number will have to increase a lot before a truly self-driving car can succeed.

 

In the meantime, deceptive and hypocritical marketing such as Tesla engages in contributes to the perception that alone among automakers, Tesla has really arrived at what the Society of Automotive Engineers calls "level 5" autonomous driving.  In a letter to California's Department of Motor Vehicles in March of this year, Tesla representatives admitted that the most advanced features of any Tesla vehicle amount only to Level 2 autonomy.  According to the SAE, Level 2 automation is simply driver-support, not automated driving, and requires that the driver constantly supervise the car's operation.  Clearly, many Tesla drivers are going beyond that.  Dr. Varner gambled on getting away with it and lost.

 

There's nothing new about automakers providing features on cars that some drivers abuse.  The muscle cars of the 1960s had power and acceleration that went way beyond anything normal driving required, and as a consequence some people wound up dying in fiery crashes after 140-mph joyrides.  But bigger engines and faster cars were just incremental changes that took place since the invention of the automobile.

 

Cars that seem to drive themselves are something truly new in automotive history, and we are still in the early stages of seeing how autonomous driving will play out.  While Tesla deserves credit for marketing what is probably the most technically advanced combination of driver-assist technologies on the market today, they have created a dangerous situation in which even a few spectacular crashes such as the one in Houston can put a damper on an entire technology and scare people away from it. 

 

If Tesla is smart enough to make a nearly autonomous car, they are also smart enough to figure out how to keep drivers from absenting themselves from the wheel until the future date when it is reasonably safe to do so.  By Tesla's own admission, that date is not here yet, but it is apparently ridiculously simple to fool a Tesla car into driving itself while you play your violin in the back seat.  Other car makers are taking more sophisticated precautions such as eye-motion detection to ensure that the driver-assist system always has a driver to assist who is paying attention.  It's way past time for Tesla to do something like the same.

 

Sources:  I consulted the following articles in the preparation of this blog:  "Feds investigating fatal Tesla crash in Texas" (Austin American-Statesman, 4/20/2021), "Tesla crash shows Autopilot isn't there yet," (Austin American-Statesman, 4/22/2021), Car and Driver, https://www.caranddriver.com/news/a36189237/tesla-model-s-fire-texas-crash-details-fire-chief/,

Reuters at https://www.reuters.com/business/autos-transportation/us-probes-fatal-tesla-crash-believed-be-driverless-2021-04-19/, the New York Post at https://nypost.com/2021/04/21/victims-in-deadly-houston-telsa-crash-identified/, Click2Houston.com at https://www.click2houston.com/news/local/2021/04/18/2-men-dead-after-fiery-tesla-crash-in-spring-officials-say/, and the Society of Automotive Engineers at https://www.sae.org/news/press-room/2018/12/sae-international-releases-updated-visual-chart-for-its-%E2%80%9Clevels-of-driving-automation%E2%80%9D-standard-for-self-driving-vehicles. 

Are Self-Driving Cars More Dangerous?

Around midnight Sunday, Dec. 29, 2019, the driver of a Honda Civic headed northbound on Vermont Avenue in Gardena, California was making a left turn from Vermont onto Artesia Boulevard.  The traffic light at the intersection was red for westbound traffic on Artesia, and the intersection happened to be the western end of the Gardena Freeway, where it becomes a surface road.  At that moment, a Tesla Model S zoomed off the freeway westbound through the red light and crashed into the Honda, killing its two occupants.  The Tesla driver and his passenger were not seriously injured.  Early news reports failed to indicate whether the Tesla was on autopilot, but National Highway Traffic Safety Administration (NHTSA) officials are investigating the crash to determine whether the autopilot was engaged.

This latest fatality involving an autopilot-equipped Tesla inspired an Associated Press review of recent fatalities involving Tesla cars in which the autopilot was engaged.  The curious reader can view the website www.tesladeaths.com, where someone has attempted to compile a complete list of worldwide statistics for fatal crashes involving Tesla cars.  As of the end of 2019, the list totaled 110 deaths, of which only 4 are in the category of "verified Tesla autopilot death."  As well over 200,000 Teslas have been sold, these statistics are not particularly remarkable, except for the fact that Tesla purports to be the leading edge of the automotive future.  As such, it deserves closer scrutiny, and that is what it's getting.

The problem in answering the question of our headline is, "more dangerous than what?"  Not only is Tesla the world's best-selling plug-in passenger car, it offers what many regard as the most sophisticated commercially available autopilot system as well.  And in contrast to the more conservative approach many automakers have taken in adding self-driving features such as lane following and automatic braking, the Tesla driver can turn on autopilot and let go of the wheel.  Such behavior is not advised by Tesla, but since when have instruction manuals been 100% effective in keeping people from doing stupid things?

The Associated Press article quotes Jason Levine, head of the nonprofit Center for Auto Safety in Washington, as saying, “At some point, the question becomes: How much evidence is needed to determine that the way this technology is being used is unsafe?”  Levine criticized the NHTSA for dragging its feet instead of issuing regulations as to how Tesla's autopilot feature can be used.  Simply warning the driver that he or she should be alert at all times when the autopilot is working doesn't make it happen.  At least two fatal U. S. crashes (one in Florida and one in Ohio) happened when the autopilot's sensors became confused and failed to recognize a large truck blocking the roadway.  Presumably, if the drivers had been paying attention, they might have seen the truck and stopped.

Promoters of the autonomous-vehicle future face two distinct but interrelated obstacles that could delay or even prevent widespread adoption of self-driving cars.

The first obstacle is technology.  The Society of Automotive Engineers (SAE) has defined five levels of automated driving.  Level 0 is a 1955 Plymouth—completely manual operation—and Level 5 would be the equivalent of an electronic chauffeur—the passenger can watch TV, sleep, or do anything else you would do if you knew a trusted and competent human driver was in charge.  No automaker currently offers a Level 5 vehicle for sale, but Elon Musk is claiming that early this year, Tesla will start to sell fully self-driving cars, which sounds like Level 5 to me.  Of course, this may be nothing but vaporware.  But unless some pretty radical improvements are made in autopilot technology, it's inevitable that some fatal crashes will happen in which the autopilot was engaged.

And here's where we run into the second obstacle:  public perception, including the perception of government regulators, lawmakers, and their constituents.  I don't know about you, but I would feel a lot worse thinking about dying in a car wreck in which my car was driving itself, rather than dying in one where I was actively at the wheel.  True, I'd be just as dead in either case, but there's something about the hope that one can make a difference if one is trying to control the situation.  This is not a completely rational state of mind, but carmakers learned decades ago to appeal to the sub-rational "lizard brain" of the consumer.  Why else do pickup ads show their products bounding over rugged mountains and doing extreme feats that 99% of drivers will never have to do? 

The budding autonomous-car industry is still treading on very shaky ground, at least in the U. S., where the majority of fatal accidents involving Teslas have occurred.  As the statistics show, less than 10% of fatal crashes involving Teslas are associated with the use of the autopilot.  But statistics do not count for much in public perception, and Elon Musk's cowboy-style reputation lends credibility to the accusation that he and his company are playing games with the safety of their customers, and by implication, with the safety of anyone within collision range of a Tesla.

If properly designed and deployed, I concede that autonomous vehicles could lower the rate of traffic fatalities while lessening traffic congestion and doing other good things such as reducing carbon emissions as well.  But there is a world of challenges in that "if."  There may be unknown factors that no one will discover until there is a certain critical mass of autonomous vehicles on the road already.

In statistical mechanics involving solutions of, say, salt in water, you can apply simple rules to very dilute solutions, so dilute that each salt molecule or atom can be treated like it is the only one in the solution.  So far, autonomous vehicles are so rare that each one is surrounded by a sea of non-autonomous vehicles, and the software probably operates under that assumption.

But when you put enough salt in the water so that each salt atom gets within shouting distance of another salt atom, things get complicated.  New effects such as saturation and crystallization occur, and your analysis has to be more sophisticated in order to deal with these effects. 

If autonomous vehicles, especially those made by different manufacturers, ever become common enough so that one vehicle can "see" another one in a typical driving situation, it is very likely that novel and perhaps hazardous effects will occur that even the designers may not have anticipated.  But that will never happen if the public gets so fearful of accidents involving autopiloted cars that they are regulated out of existence.  I hope that doesn't happen either, but if Musk and Tesla get too careless, they might end up triggering just such a reaction.

Sources:  The Associated Press article I referred to appeared in many locations, among which was the San Jose Mercury-News website at https://www.mercurynews.com/2020/01/03/3-crashes-3-deaths-raise-questions-about-teslas-autopilot/.  I also referred to the same site for information on the Gardena crash at

 https://www.mercurynews.com/2020/01/02/fatal-tesla-crash-in-california-investigated-by-feds/.  The Tesla fatal-crash statistics website is www.tesladeaths.com, and I also referred to the Wikipedia article on Tesla, Inc.

Self-Driving Car Fatality No. 1: Joshua Brown Makes History

On May 7 of this year, Joshua Brown, owner of a wireless-network technology company and Tesla car enthusiast, was riding in his Tesla Model S on a divided highway in Florida.  Mr. Brown loved his car and posted numerous YouTube videos that showed him using the autopilot function in the "look, Ma, no hands!" mode.  By all accounts, Brown was a generous, enthusiastic risk-taker (his specialty when he was in the military was disarming weapons, according to a New York Times report), and hands-free driving went against the explicit instructions Tesla provides for the autopilot feature.  But Tesla owners do it all the time, apparently, and until May 7, Mr. Brown had gotten away with it.

Then a tractor-trailer rig made a left turn in front of Mr. Brown's Tesla.  According to a statement by Tesla, the high ground clearance of the trailer and its light color, resulting in low visual contrast against the sky, failed to trigger the car's brakes.  The Tesla ran underneath the trailer, fatally injuring Mr. Brown.  A neighbor quoted Mr. Brown afterwards as saying in another context a few weeks before the accident, "For something to catch Elon Musk’s eye, I can die and go to heaven now."  No one knows how serious Mr. Brown was when he said that.  But he will go down in history as the first person in the U. S., and perhaps in the world, to die in a car that was operating in its self-driving mode.

Will this tragedy spell doom for self-driving cars?  Almost certainly not.  The first recorded steam-locomotive railway fatality was that of the English politician William Huskisson, who attended the opening ceremonies of the Liverpool and Manchester Railway on Sept. 15, 1830, which featured inventor George Stephenson's locomotive the Rocket.  Wanting to shake the hand of his former political enemy the Duke of Wellington, Huskisson walked over to the Duke's railway carriage, then saw that the Rocket was bearing down on him on a parallel track.  He panicked, tried to climb onto the carriage, and fell back onto the track, where the locomotive ran over his leg and caused injuries that were ultimately fatal.  Passengers had been warned to stay inside the train, but many paid no attention. 

If Huskisson's death had been mysterious and incomprehensible, it might have led to a wider fear of railways in general.  But everyone who learned of it took away the useful lesson that hanging around in front of oncoming steam locomotives wasn't a good idea, and railways became an essential feature of modern life.  Nevertheless, every accident can teach engineers and the rest of us useful lessons in how to prevent the next one, and the same is true in Mr. Brown's sad case.

It's not clear how long the Version 7.0 of the Model S software featuring the autopilot function has been available, but it's probably been out for at least a year.  Multiply that time by the number of Model S owners and how far they drive, and you have a track record that shows if anything much is wrong with the software, it's not very wrong.  Model S owners aren't dying like flies in autopilot accidents.  Still, telling drivers how great a self-driving feature is, and then expecting them to pay constant attention as though the car were a driver's ed student and you were the instructor, is sending a mixed message.

Tesla's own posting about the accident cites statistics that show if anything, Model S cars have a lower accident rate than average, and that may be true.  But as Tesla's public profile rises, the firm has some delicate maneuvering ahead of it to avoid becoming a target for lawyers who will want to portray Tesla in court as heedless of driver safety.

We've known since the earliest days of automobiles that they are dangerous in careless hands and require constant vigilance on the part of the operator.  Plenty of people ignore that fact and pay for it with injuries or their lives, and take the lives of others as well.  But everybody, whether safe or careless, still admits it's a good idea to pay attention while you're driving.

Now, however, something fundamentally new has been added.  When a car has a self-driving feature that nevertheless requires you to be ready to take command at a moment's notice, the driver is torn between letting the machine take over and keeping a constant lookout for trouble.  You can't both be constantly vigilant and also watch a Harry Potter movie, as Mr. Brown may have been doing at the time of the accident.  In most of us, especially guys, attention is a focused thing that has to be directed at one primary target at a time.   Even if I had a self-driving car (which I don't), and after driving it for a while and learning what it typically can and can't do, I wouldn't feel very comfortable just sitting there and waiting for something awful to happen, and then having to spring into action once I decided that the car wasn't doing the right thing.  That's a big change of operating modes to ask a person to do, especially if you've been lulled into a total trust of the software by many miles of watching it perform well.  Who wouldn't be tempted to watch a movie, or read the paper, or even sleep?

I'm afraid we've got some institutionalized hypocrisy here that most auto companies are fortunately free of.  But Tesla is a different kind of beast, founded at a time when anybody who ever installs software is either forced to lie, or actually has to read dozens of pages of legal gobbledegook before clicking the "I Agree" button.  The impression I have of the arrangement between Tesla and Model S owners is that Tesla pretends that  they have to keep their hands on the wheel, and the owners pretend that they're following instructions.  And the pretense has made the lawyers happy, I suppose—until now.

Now that the much-anticipated First Fatality has happened, things could go in any of several directions.  The National Highway Transportation Safety Administration, which is investigating the accident, could come out with a bunch of heavy-handed federal regulations that could squash or set back autonomous vehicles in the U. S. for many years.  Joshua Brown's relatives could mount a lawsuit that could cripple Tesla.  Or (and this is the one I'm hoping for), Tesla's engineers can learn what went wrong in Mr. Brown's case, fix it, and deliver clearer, more practical instructions to drivers, including some human-factors engineering that seems to be missing, about how to use the self-driving feature, so that the remaining Tesla drivers can lessen their chances of becoming Fatality No. 2.

Sources:  Many news outlets carried reports of Mr. Brown's death.  Tesla's own posting concerning the incident appeared June 30 at https://www.teslamotors.com/blog/tragic-loss.  I referred to reports on Fortune's online version at http://fortune.com/2016/07/02/fatal-tesla-crash-blind-spot/, the New York Times report on Mr. Brown's background at http://www.nytimes.com/2016/07/02/business/joshua-brown-technology-enthusiast-tested-the-limits-of-his-tesla.html, the Tesla press kit on its autopilot at https://www.teslamotors.com/presskit/autopilot, and the Wikipedia article on William Hoskisson.  Thanks to my wife for notifying me about the incident.