Monday, December 30, 2019

Boeing Chief Fired Over 737 Max Controversy

On Sunday, Dec. 22, members of the board of directors of Boeing held a conference call and decided to fire Boeing CEO Dennis Muilenburg.  Since the grounding of the company's 737 Max jetliners last spring after two crashes that killed over 300 people, Muilenberg has faced increasing criticism.  At issue is the jetliner's Maneuvering Characteristic Augmentation System (MCAS), a software patch that was intended to make the 737 Max fly more like its predecessor airframes, which date back to the 1960s.  But in documents released last October, Boeing's former chief test pilot Mark Forkner wrote in an email as long ago as 2016 about "egregious" behavior of the MCAS in flight-simulator tests.

Leaders in an engineering-intensive industry face constant conflicting pressures.  On the one hand, there is the need to make a profit so that your organization can continue its existence and benefit the public in some way with its products and services.  On the other hand, demands for resources to ensure safety and reliability of those products and services cost money, and the trick is to strike a balance between excessive engineering that runs profits into the ground, and skimping on due diligence that leads to shoddy products.  Not being qualified to run a lemonade stand myself, I have nothing but admiration for executives who manage this balancing act, and until recently, Dennis Muilenburg was apparently doing it well enough for the Boeing board of directors to keep him on.

But no longer.  After the fatal 737 Max crashes in Malaysia and Ethiopia were shown to be due to unexpected actions of the MCAS, both the U. S. Federal Aviation Administration (FAA) and eventually the U. S. Congress began investigations into the development of the aircraft and the reasons why MCAS was designed in the first place.  As we mentioned in an earlier blog, a series of physical design changes involving bigger engines made the 737 MAX airframe behave very differently than its predecessors.  According to Gregory Travis, a software engineer and pilot who examined the issue, the right thing to do at this point was for Boeing to undertake a complete mechanical redesign of the aircraft, which would have been very costly in terms of both time and money.  Instead, Boeing chose to create a software patch—MCAS—that sought to make the plane handle more like it used to handle.

The problem was that under some combination of instrument failures, MCAS drew the wrong conclusions about what was going on with the plane, and took over the flight controls from the pilots in a way that was both startling and extremely difficult to overcome.  The Malaysian and Ethiopian crews were not able to do this, and their planes crashed. 

At first, Boeing blamed inadequate pilot training for the crashes, but as the firm has released more internal documents in response to Congressional inquiries and FAA requests, it's beginning to look like at least some people inside Boeing had grave doubts about the viability of the MCAS for safe flying.  Although the public has not yet obtained access to most of these documents, some emails released in October reveal that back in 2016, test pilot Mark Forkner had doubts about the MCAS even when it was only incorporated into the controls of a flight simulator.  The U. S. House committee familiar with the documents says that "the records appear to point to a very disturbing picture of both concerns expressed by Boeing employees about the company’s commitment to safety and efforts by some employees to ensure Boeing’s production plans were not diverted by regulators or others."

An organization's culture is one of the hardest things to describe, but it can be one of its most important assets, or just as easily a liability.  In the quasi-military structure of most commercial firms, leadership sets the overall tone of a culture, but it's a constant struggle to maintain that tone throughout all parts of the organization. 

"Transparency" is a word that shows up a lot when a firm like Boeing appears to have been concealing information that might have made it look bad, or caused regulatory problems and delays in production.  Obviously, transparency is a relative goal.  No firm in a competitive market can afford to be completely transparent about its plans and specialized technologies.  At various times, engineering-intensive companies have tried this in the form of technical newsletters, in which their engineers bragged about their latest developments in enough detail to allow competitors to copy and improve upon them.  Needless to say, such newsletters are found today only in the dusty shelves of libraries that keep material from defunct companies, such as General Radio and the original incarnation of Hewlett-Packard. 

But transparency is a necessity when it comes to issues that affect safety.  On an individual level, the moment you feel a need to hide something you're doing, this can serve you as an alarm to lead you to question why you're hiding it.  But in an organization in which the immediate pressures tend to be in favor of shipping products and minimizing any issues that would stand in the way of that goal, it's easy to simply not say something you ought to say, or not deliver the bad news that will disrupt the schedule that marketing wants to keep. 

The buck stops at the CEO's office, and in firing Muilenburg, Boeing's board of directors has acknowledged that the company's culture has to change from the top down.  Whether a new leader can take the company back to a point where its 737 MAX jetliners can be flown safely again is still very much an open question, however.  Scrapping them or recalling them for a major mechanical redesign would probably spell an end to Boeing as a commercial-aircraft firm, leaving the field to Airbus.  But it's hard to see how anyone is going to have a great deal of confidence in a fix that is mainly software, which is how the 737 MAX got into this mess in the first place. 

Monday, December 23, 2019

Safe People or Safe Systems? The Ring Security Breach

On Wednesday, December 4, eight-year-old Alyssa LeMay heard the sound of Tiny Tim singing "Tiptoe Through the Tulips" coming from her bedroom upstairs in her home in Mississippi.  As she walked into the room, the music stopped and she heard a voice say, "Hello there."  As she looked around the room to see where the voice was coming from, it called her a racial slur which was neither acceptable nor accurate, claimed that it was the voice of Santa Claus, and told her to start misbehaving by, for example, breaking her TV.

Having more sense than to listen to such temptations, she went downstairs and told her father, "Someone's being weird upstairs."  He discovered that a Ring security camera that the family had bought during a Black Friday after-Thanksgiving sale had been taken over by someone who obviously wasn't supposed to be able to do that. 

The LeMays eventually contacted the Washington Post, whose story on the episode was republished widely.  When the LeMays called Ring to complain, they were told basically that the breach was their fault.  Ring determined that the bad actor had obtained the LeMay's username and password from another site and used them to hack into Alyssa's bedroom.  Ring castigated the LeMays for not using the two-step authorization method that Ring recommends.  In a statement published on Ring's website, the company said "we have investigated this incident and have no evidence of an unauthorized intrusion or compromise of Ring’s systems or network."

Let's step back a moment and parse that statement.  What Ring means by unauthorized, and what the LeMays mean by unauthorized, appear to be two different things.  Only an authority, an entity or person capable of authorizing someone, can really authorize an intrusion or compromise.  For that matter, saying "unauthorized intrusion" is like saying "impermissible burglary."  I'm not aware of any kind of burglary that is permissible, or an intrusion that is authorized.  But the point is that the LeMays were by any reasonable standard, the only people who are logically empowered to authorize access to the camera, microphone, and speaker in their daughter Alyssa's bedroom.  They did not authorize the criminal who gained access to the Ring device, and therefore, by this reasonable, common-sense definition of "authorized," there was unauthorized access.

Now look at it from Ring's point of view, which by implication is Amazon's point of view, as Amazon owns Ring.  Think like a software lawyer for a minute.  When we sell a product to a consumer, we have to make sure that the consumer has enough information to avoid problems with the product.  We as lawyers observe the legal fiction that every one of our customers always reads all the fine print and boilerplate that comes with all our products, including the stuff about installing two-step verification for passwords, using strong passwords, and so on.  If we actually made the product so that it wouldn't work unless the user really took all these complicated measures, very few people except computer nerds and lawyers would buy it, so we make it so it will work even if you leave your username as "1234" and your password as "password."  But if the user is so negligent, stupid, (fill in your favorite lawyerly pejorative adjective here) as to not take the recommended precautions, well, too bad.  We've done our lawyerly job, and if anything goes wrong it's on the consumer's head.  To us, "unauthorized" means that somebody hacked into our system and was able to access a device that even the most computer-savvy consumer installed with all the security bells and whistles.  And that didn't happen here, so we are blameless.  Legally speaking.

There is a progression in the safety and security of innovative technologies that often follows a well-known pattern.  At first, a new technology requires the users to learn lots of detailed precautions that must be followed to avoid injury or other types of harm.  But as the technology becomes more widespread and lesser-trained people use it, the harms that can come from uneducated users sometimes happen more often, so often that the very existence and continued use of the technology is threatened.  Only then will the technology's designers step back and ask themselves, "How can we make this really foolproof, so that someone who knows next to nothing about it can nevertheless use it safely?"  At that point, engineers begin to design safety into the technology itself.  It may cost a little more, but the improvement in safety when used by untrained personnel is usually worth it.

This pattern happened with railroading, it happened with automobiles, and in some ways it's happened with computer and information technology.  But not nearly enough, as Alyssa's story shows.  In consumer electronics, where ease of use and cheapness are two paramount requirements, security often becomes an afterthought.  A non-technically-trained user who simply wants to be able to check on his or her daughter with a camera should not be expected to do anything that isn't strictly necessary to set up the system.  The two-step verification security precaution obviously wasn't necessary for the camera to work, so the LeMays didn't do it.  And by reusing passwords—an unfortunate but understandable practice in these days of seventeen gazillion passwords that all our devices and services demand of us—they created a situation in which some hacker stole their credentials and used them to access the Ring device in Alyssa's room.

Ring wants their consumers to be safe people—people who don't reuse passwords and who read enough of the fine print in the online instructions to go the extra mile and install extra, though non-necessary, security precautions.  But people, by and large, want safe systems—systems that simply will not work unless they are set up with sufficient security to begin with.  And history shows that the systems and technologies that survive beyond a highly trained niche market are usually safe systems—systems that anybody off the street can get running with a minimum of effort without running the risk of endangering himself, herself, or one's family members. 

Sources:  The Austin American-Statesman carried the Washington Post's article "Camera in child's room hacked, 8-year-old harassed" on pp. E3-E4 of their Dec. 15, 2019 edition.  The statement from Ring concerning this incident can be found at

Monday, December 16, 2019

Climate Change and Attitude Change

The old joke about how an optimist and a pessimist can see the same glass of water and say different things about it applies to a lot of things.  The optimist who says it's half full brings a different attitude to the same physical facts that the half-empty pessimist looks at, but draws different conclusions from them. 

Climate change and the effects of rising carbon dioxide levels on global temperatures and weather have led to a widespread attitude of despair, according to Matt Frost, a policy analyst who recently published an article called "After Climate Despair" in The New Atlantis.  His approach to climate change is neither denial nor agreement with the prevailing consensus of certain political groups that we are staring doom in the face.  Instead, it's a good example of how attitude can make a big difference in the interpretation of facts.

The standard high-level public-policy take on climate change goes something like this.  Humankind has foolishly burned itself into an ongoing crisis that will, if not averted by radical and draconian imposition of fossil-fuel bans and restrictions, lead to the downfall of civilization and the destruction of the ecosphere.  The only viable solution is the imposition of a global austerity plan that rolls back global energy use to a level comparable to what it was back somewhere in the 19th century, and even then, it will take decades or centuries before any notable improvement will come.  The fact that the major world governments have not fallen into line and cooperated with this solution is cause for despair, a despair akin to that which relatives of a hopeless drug addict feel when they try to intervene, but the addict goes right on shooting up until he overdoses.

Frost begins by distinguishing between the main factor in climate change—namely, the burning of fossil fuels that increase the levels of carbon dioxide in the atmosphere—and the fact that energy abundance and growth is necessary for human flourishing in today's world.  Perhaps the key insight he brings in his set of proposals is that we should look on carbon dioxide emissions not as a horror to be avoided at all cost, nor as totally innocuous, but as waste, similar to sewage, scrap iron, or other byproducts of industrial activity that engineers have learned  how to deal with in the past. 

He examines several proposed solutions that would reduce global warming, and discards them for various reasons.  Switching to burning wood instead of coal and oil and gas is impractical because it would require huge amounts of farmland that we don't have, or that we need already for food.  Throwing tons of sulfur dioxide into the atmosphere to reduce the influx of infrared radiation, while possibly reducing global temperatures, might screw up the ecosphere even worse than it is now.  The basic approach he recommends is one of energy abundance, and we have plenty of knowhow to bring that about with only minor changes in directions that we're already pursuing.

For one thing, nuclear energy is sadly underutilized in most countries, a notable exception being France.  While nuclear waste is a problem, it's a localized manageable problem and doesn't automatically escape into the air and cause climate change.  Treating carbon dioxide emissions as a waste product similar to sewage would allow the sensible, deliberate implementation of regulations backed by engineering solutions that might lead to sequestering or reuse of the gas, which after all, given sufficient energy, can be reconverted into fuel again.  While such processes are done only on a pilot scale today, if we realize that lower energy prices would make them more practical, we could break through the barrier of despair and do something about carbon in the atmosphere by means of the very energy that the present despairing attitude would have us say good-by to.

Another good point that Frost makes is that these sorts of things can be done on a small scale:  a solar installation here, a carbon-abatement plant there.  These sorts of things don't need any giant global bureaucracy to administer.  While dealing with the present and future consequences of climate change will present challenges that are in some ways unique, throwing up our hands and giving up on civilization is not the answer.  In a sense, engineering got us into the situation we're in today, and as long as we believe engineering can help us deal with the consequential problems, we have a handle on possible paths to solutions.   

As I read Frost's article, it occurred to me that some of what he's proposing has already taken place.  Watt for watt, burning natural gas for energy produces less carbon dioxide emission than burning coal.  An early worrier about climate change in, say, 1990, might have come out in favor of a massive government-directed effort to shut down all our cheaply operating coal-fired power plants and force them to burn expensive natural gas, at the price of raising electricity prices by 300% and causing a recession.  

That didn't happen, but something else did that bureaucrats didn't expect.  The petroleum industry developed fracking and a spectrum of other technologies that led to the exploitation of abundant natural-gas reserves in old oil fields, which has sent natural gas prices plummeting and shuttered coal-fired plants, not because they're illegal, but because they're unprofitable.  As a result, the U. S. power-generation industry is now more carbon-friendly than it used to be as a whole, all without heavy-handed government intervention.

We can't rely on the market to pull this kind of benign trick all the time, but it's an example of how a can-do optimistic attitude toward a difficult situation can lead to surprisingly good results.  Perhaps not all of Frost's specific policy proposals will find favor in the halls of power, but what I hope people do take from him is his attitude.  In the Roman Catholic catalog of sins, despair is the one unforgivable sin, because by definition, if you give up hope of salvation, you can't be saved.  The principle has applications beyond theology.  If we decide that the only way to reduce carbon emissions is to achieve the politically impossible, well, by definition, that's not going to happen.  Frost's advice is to look at the wide array of possible and even local things we can do, and work on those.

Sources:  Matt Frost's article "After Climate Despair:  Embracing Abundance in a Warming World"  appeared in the Fall 2019 issue of The New Atlantis, pp. 3-21.  I also referred to Mr. Frost's webpage at, where from his resumé I learned that he has five children, and is thus invested in seeing the future turn out better than it might.