Monday, October 29, 2018

Open Loop in Lawrence, Massachusetts—Cause of the Columbia Gas Disaster

Every profession has its inside lingo, expressions that mean something only to practitioners.  In the disciplines of electrical and mechanical engineering, one such expression is to "go open loop."  The loop referred to is a feedback loop, a concept that control systems of all kinds use to regulate quantities such as speed, flow rate, and pressure.  A well-designed feedback loop works as a clever automatic hand on a control valve to maintain constant pressure in a gas supply line, for instance, in the face of variations in demand for gas or delivery pressure from the high-pressure gas main.  But in order to work, the feedback loop must be closed, or complete, all the way from the regulator that controls the pressure, say, through supply pipes to the sensor that tells the system what the pressure is, back to the regulator.  If the information flow is interrupted anywhere in the loop, the regulator usually goes to an extreme and jams, which can lead to dire consequences.  So if you ever hear an engineer talking about a person who lost control of his temper as "going open loop," that's where the expression comes from.

On Sept. 13 of this year, dozens of people were injured, one was killed, and over a hundred structures were damaged in and around Lawrence, Massachusetts, when a natural-gas supply line's pressure soared and turned pilot lights into blowtorches and stove gas burners into towering infernos.  The National Transportation Safety Board has released its preliminary report on the cause of the disaster, and it looks very much like it was a classic case of open loop.

That afternoon, work crews were in the final stages of performing a tie-in to connect a new set of plastic gas-distribution pipes to the system, and to decommission a set of old cast-iron pipes that dated back to the early 1900s.  While many modern gas distribution systems place individual regulators at each customer's location, the older systems such as the one in Lawrence used low-pressure gas (about 0.5 pounds per square inch gauge, or psig) in the distribution pipes.  To regulate this pressure, control-system sensors were placed on the pipes and fed their data back to regulators at the junctions between the high-pressure transmission pipes and the low-pressure distribution pipes.

The instructions to the work crews did not say anything about what to do with the sensor  signals coming from the old cast-iron pipes when the gas was switched over from the old system to the new plastic one.  So when the crews shut off the manual valve that fed the old pipes, the sensors in them were still connected to the regulator that was now feeding the new plastic pipes, which were connected to most of the homes in the affected area. 

It's easy to see that cutting off the valve broke the feedback loop.  The control system saw pressure falling in the old pipes, so it opened up the regulator.  But the higher pressure wasn't being sensed, because the crews hadn't been told to switch the sensor signals at the same time they switched the pipes. 

Pressures are monitored constantly on the system, but only at Columbia Gas headquarters back in Columbus, Ohio.  Overpressure alarms went off shortly after 4 PM at the monitoring center, but the dispatchers there had no way of shutting off the system.  All they could do was to try getting in touch with the technicians in Lawrence to tell them to shut off the gas.  Even though that was done about a minute after the alarms went off, it wasn't until 4:30 that the regulator supplying the overpressure gas was shut off.  By then most of the damage had been done. 

An article in the Boston Globe last week describes the anguish of Lawrence residents who are still waiting for Columbia Gas to restore gas service.  Many homes in the area are unlivable without gas heat, and as cold weather intensifies the situation is getting intolerable.  The company's self-imposed deadline of mid-November for having all customers back on line recently slipped to Dec. 16, and there are many complaints about poor communication regarding repair-work schedules and shoddy workmanship once the crews arrive.  Columbia Gas is going to be paying for this mistake for a very long time, as are the affected residents of Lawrence.

The behavior of Columbia Gas approaches a poster-child status of how not to run a utility.  You can bet that when the gas company was locally owned and operated, the monitoring center in Massachusetts probably also had the ability to shut off the gas.  But when consolidator Columbia Gas purchased the system, the move of the monitoring operation to Ohio was probably done to save money, money which was not spent on a corresponding control system to allow Ohio monitors to shut off regulators and valves remotely. 

Work-crew training and planning also came up short in this disaster.  It's sometimes hard to say exactly how much knowledge technicians and their immediate supervisors should have about the technology they are working on.  There's no need for every lowly tech to know enough to design the entire system, for example. That's what engineers are for.   But on the other hand, you would hope that the person in charge of the work crew would know enough to realize that if the sensor signals were not swapped at the same time as the pipe systems they were connected to, there could be trouble.  Presumably the company has done this sort of thing before successfully, so it may be a case of an isolated incompetent worker rather than a systemic issue.  But clearly, more explicit instructions and better training are needed.

Finally, Columbia Gas hasn't exactly covered themselves with glory in the public-relations department.  Gas utilities are by nature local monopolies, and it is easy for them to act like what they are, namely the only gas company in town.  Unfortunately, in misleading their customers as to when gas would be restored, they have created a lot of ill will that more careful and cautious scheduling could have avoided. 

Let's hope for Lawrence's sake that it's a mild winter in Massachusetts, and that everybody gets their gas lines working again before it's too much colder.  And for all you other gas utilities out there:  don't act like Columbia Gas.

Sources:  I referred to the article "Merrimack Valley residents voice frustration with recovery effort" which appeared on the Boston Globe website on Oct. 28, 2018 at  The preliminary NTSB report on the incident is available at  I blogged on this disaster on Sept. 17, 2018.

No comments:

Post a Comment