Monday, October 29, 2007

Working the Bugs Out In Space

If you see metal shavings in the oil you change out of your car, that's not an encouraging sign. But what if your vehicle cost several billion dollars and is flying hundreds of miles above the ground at fifteen thousand miles an hour? That is the problem faced by the engineers and astronauts trying to build the International Space Station.

News reports this week say that space shuttle Discovery mission specialist Daniel Tani opened a plastic cover on a gearbox during a spacewalk to reposition some solar panels. He was following orders from ground engineers who had noted excessive vibration and power consumption from the motors that move the 30,000-pound solar panels so as to collect the maximum amount of sunlight. Inside the box, Tani found an abundance of metal shavings, and collected some for analysis back home.

Everything is harder in space: repairs, inspections, lubrication, and even engineering and design. Although there are a few expensive giant vacuum chambers around that let engineers test satellites and other small to medium-size objects in something close to the reality of space, these don't simulate zero-G conditions. So the only way to check out most space-bound systems in 100% realistic conditions is to fasten them on a rocket and send them out there to see what happens. This is one reason that space exploration is so expensive and fraught with failures.

Readers of this blog know that I have serious reservations about the continuing use of the Space Shuttle (it ought to be replaced yesterday, not in two or three years) and the wisdom of spending billions on a space station which is too shaky for really good science and too small for really meaningful colonization of space. All the same, it's good to know that when something goes wrong on a system as big as the Space Station, you can send up a guy to take off the covers and have a look around, even if the service call costs millions of dollars. Discovery's latest trip was not only for maintenance—it is part of a tightly scheduled program to keep the Space Station's construction on track for completion by 2010.

Since this effort is costing several countries (Russia, the U. S., Japan, and Canada are major partners) both money and lives (if you count those who died in the 2003 Columbia disaster), it is only reasonable to ask what good it is doing. There is a scientific answer, an engineering answer, and a political answer. As is the nature of these things, they all blur into each other.

The scientific answer is, so far, not much. I cannot think of a single major scientific discovery that has resulted from work performed directly by astronauts, as opposed to research enabled by the Hubble Space Telescope or other unmanned lunar and planetary probes. This of course may change once the station is "completed" (such a project is never really finished for good, but the bulk of work will eventually shift from construction to use). But right now, it's too early to say if there will be any significant scientific payoff from the project at all.

From an engineering standpoint, building and operating the space station can tell us loads about the problems of building and operating a space station. We've had a smoke problem, a computer problem, and now a ground-up-gear problem, possibly, and those are only the ones that made headlines. As the first system of its kind, the International Space Station is bound to have all kinds of engineering issues that we can learn a lot from, assuming we try to do something like this again. As every engineer knows, the first time is mainly learning from mistakes. If your funding goes long enough to let you try a second time, you have a chance at getting it mostly right.

From a political view, the space station is an experiment in international cooperation on an intensely complex technical project, and by and large, this aspect of it seems to have gone well. When the U. S. manned space program went on hold for two years after the Columbia disaster, the Russians stepped up to the plate and kept the station in business with Soyuz launches. So far, the politicians have mostly kept out of the way of the committed engineers and managers in all the countries involved who want to see this thing go. Engineers have a way of forgetting about nationalities or political differences when they share a common technical goal, and the International Space Station is a good example of how that can work.

In the meanwhile, there's the question of where all those metal shavings are coming from. The ten-foot boxes that serve as pivots for the large solar panels could be replaced, I suppose, but that would be a major undertaking. On the other hand, if the bearings freeze up that will severely limit the amount of electrical power available to the station. I hope this turns out to be something trivial, as one engineer on the ground hoped that the shavings were just chewed-up foil insulation. My instincts tell me that such a hope is wishful thinking, but we'll just have to wait and see.

Sources: The New York Times article describing the metal-shaving problem is at Wikipedia has a good articles on the International Space Station's history and construction.

Monday, October 22, 2007

One Laptop Per Child: Will It Fly?

Being poor and isolated is rotten. A recent book by Paul Collier entitled The Bottom Billion: Why the Poorest Countries Are Failing and What Can Be Done About It deals with the poorest one-sixth of the world's population of six billion. According to reviews, Collier identifies four main reasons that these poorest of the poor are where they are. Internal and regional conflicts (1) are sometimes worsened by concentrations of natural resources (2) such as gold and oil that distort economies, especially when (3) you live in a country next to one where similar problems are going on, and (4) your government is corrupted by sweetheart deals with everybody from Western multinational companies all the way down to international crooks. Although I haven't read the book, the problem of a country's poor children not having laptops apparently did not make Collier's list of the top four issues. Nevertheless, an organization in Cambridge, Massachusetts is busily working on solving that problem.

The outfit called "One Laptop Per Child" aims to put specially-designed, inexpensive laptop computers into the hands of millions of children in the poorest countries in the next few years. The machine itself will be powered by solar cells, hand crank, or batteries, and uses special hardware and software to reduce its operating power consumption to less than a watt under some conditions, which is about a tenth or less of what an ordinary laptop uses. Recent reports indicate that the designers have not yet reached their target cost of $100 per unit, but present estimates are below $200 and the hope is the cost will fall as manufacturing climbs the learning curve.

The project's founder is Nicholas Negroponte, who has held various positions at MIT and related organizations for many years. Negroponte, who also founded MIT's Media Lab, is a member of what one might term the MIT computer brain trust, a group of individuals including Seymour Papert and Marvin Minsky who have shaped the direction of a great deal of computer and artificial intelligence research and publicity.

Clearly, the hearts of Negroponte and company appear to be in the right place. Children don't live by bread alone, and it is a noble goal to bring the benefits of computer technology to people who are impoverished in other ways as well. The plan is to sell the laptops only to governments, which would presumably distribute the units to their citizens either free or at a heavily subsidized low cost. Although the XO-1, as it's called, will not be available for consumer purchases in general, the Wikipedia article on it reports that this Christmas, you will be able to "get one and give one": you can buy one for yourself and at the same time, donate one to a poor child somewhere.

There is a movement in engineering ethics to encourage the study of what are called "moral exemplars": people or organizations who do the right thing in engineering, furnishing good examples to the rest of us. I will say that the XO-1 project certainly has the potential to be a moral exemplar, but so far the jury is out. The organizers are still awaiting large-scale production and distribution, and until they have large numbers of units out in the field and do some studies to see how they are used, we will simply have to wait and see how the project turns out.

A few critics have pointed out that the venture is very "top-down," in the sense that a bunch of experts in Cambridge got together and designed a laptop that they thought would be good for third-world children to use. It has certainly gained Negroponte a lot of favorable media attention. For example, he introduced a kind of pre-prototype at a UN-sponsored meeting in Tunisia in 2005, sharing the platform with then-UN secretary general Kofi Annan. And judging by the specialized hardware and software, the MIT types have had a field day trying out some of their pet ideas in this thing, using it as kind of a test bed for a lot of what-if notions.

But whether the unit really meets a genuine need or truly improves the lives of children around the world remains to be seen. One concern is the fact that all the software on the unit is open-source. This is a nice gesture toward an ideal world that some people would like to live in, where all software would be open-source, but it ignores the reality that most software used by most computers today is proprietary. And if you can't run any proprietary software on these XO-1s (although users might install it after purchase, since the operating system is Linux), there is a real danger that the things may turn into just expensive toys.

Years ago, I experienced what happens when a new piece of computer hardware is launched without any software available for it. One of the leading lights in the Massachusetts computing world back then was the Digital Equipment Corporation, or DEC. I spent a good chunk of my first research dollars as a professor on a DEC computer highly recommended by a colleague who, I found later, used to work for DEC. It was a good machine hardware-wise, but as the months dragged on and nobody besides DEC developed any software for it, I found that I'd bought an expensive boat-anchor, and ended up having to buy a PC.

I hope such a fate does not await the XO-1, but surely the developers have thought of this problem in advance. Most of the world's effective software has been developed under the aegis of the free-enterprise system where people had to pay something for it. Maybe the children will surprise us and develop software on their own—the system is said to allow for this. I wish the XO-1 the best, but a community that benefits from computers is more than just the sum of software, hardware, training, and distribution. Time will tell, as it usually does.

Sources: The official One Laptop Per Child website (in English) is at The Wikipedia article about it is at I learned about the project in an article by Kirk Ladendorf in the Oct. 22 issue of the Austin American-Statesman. Collier's book was reviewed in the November 2007 issue of First Things.

Monday, October 15, 2007

Copyright or Copywrong? The Ethics of Technological Multiplication

On Oct. 5, a jury in Minneapolis fined Jammie Thomas, a 30-year-old single mother, a total of $220,000 for downloading twenty-four copyrighted songs. Thomas was the target of a lawsuit filed by the Recording Industry Association of America (RIAA) and major music labels. Although music-downloading websites have been sued successfully in the past, this is one of the first times in recent months that an individual downloader has been fined.

Let’s leave aside, if we can, the picture this story gives us of six large, wealthy corporations, and a trade association representing many more, all ganging up on a woman who is not likely to be able to pay these fines any time soon. It can actually happen that a poor person does something wrong enough to be fined a lot of money for it, if not sent to jail. But is that what happened here?

Thomas’s case is just one tip of a huge iceberg that is floating around in electronic media today: the fact that making essentially flawless copies of a digital original requires less technical resources every week. Let’s try to clarify the issues a little bit.

Even back in the Stone Age, every tribe probably had some clowns and singers that other Cro-Magnons enjoyed listening to. These prehistoric entertainers created something of value: an economic good. Elementary justice demands that the entertainers who spend time and effort practicing and performing should receive some kind of reward for their effort. In those days, it might have been an extra joint of meat from the stewpot. Whatever the reward, the performer may have insisted on it before performing. The more people his performance attracted, the more stewpots he could sample from, but before the Internet, radio, printing, or writing, his ultimate market was pretty small.

Since the invention of writing itself (probably the oldest communications technology), the reproduction of economically desirable artifacts (stories, jokes, songs, etc.) has had a technological component. But even way back at the prehistoric origins of entertainment, there were two extremes that everyone involved had to navigate between. At one extreme, the performer has an absolute monopoly: he is the only performer in the world, everybody wants to see him perform or die, and so he can charge whatever he wants. He can demand the entire wealth of the whole tribe in exchange for one performance if he wishes. This is clearly unfair to the rest of the folks, who have themselves acted unwisely in becoming such slaves to amusement.

At the other extreme, the performer himself becomes a slave: he is threatened with death if he doesn’t perform, but he gets no rewards if he does. Anybody who wants to can walk up to him and demand a performance any time, with no charge to the members of the audience. This extreme is clearly unfair to the performer, who would be better off waking up dead some day.

You’ve been waiting for the technology to come in, right? This is an engineering ethics blog, after all. Well, here it is. All that technology can do is to multiply the performer’s performance in number, magnitude, impressiveness, duration, or other ways. But without the performer, that human being who originates the thing everybody wants to see, you have nothing. Printing, radio, television, motion pictures, phonographs, DVDs, the Internet, YouTube—all these things just give more people access to the performance, whatever it is. Now, it takes a certain amount of time and money to execute this multiplication—call it the marginal resource cost. What has happened over the last few decades is that the marginal resource cost for multiplying the performance has shrunk by many orders of magnitude. When you compare what the Bell System charged a major TV network in 1955 to operate its network transmission facilities (and factor in inflation)—probably the equivalent of many millions of dollars today—with what it costs some 14-year-old kid in Casper, Wyoming to make a video and put it on YouTube, you get some idea of how these marginal resource costs have collapsed. With some exceptions, the direction the technology has moved is to make more stuff available, for everybody, cheaper. So if there were no copyright laws at all, you’d get a situation in which few people would bother to do anything very good that requires a lot of resources (personal or financial), because they could never recoup their investment.

On the other hand, strong-arm tactics like the RIAA lawsuit against Jammie Thomas attempt to move things in the other direction, toward total, perpetual control of the performance by those who own it (not necessarily those who actually did it in the first place). Many people, including Stanford law professor Lawrence Lessig, think we have already gone too far in this direction, at least on paper. Copyright terms have been extended greatly in the last few years, to the point where many artists are worried that quoting or citing anything more recent than 1910 in print, music, or film will make them liable to a lawsuit. Part of this trend, no doubt, arises from a fear on the part of corporate copyright owners that if they don’t do something quick, everybody will digitize everything and just swap it around forever without anyone making a dime off any of it. These fears are no doubt exaggerated, and another part of the trend arises from a much simpler cause: greed.

Mixed up in all this are things like cultural traditions, expectations of private purchasers of entertainment media, technical standards and compatibilities, and many other factors which make copyright law such a happy hunting ground for lawyers. Certain acts of technological duplication in themselves should be made illegal. I don’t think anyone seriously disagrees with the principle that counterfeiting money should be against the law, even if you do it just to have some pretty pieces of paper to look at and you never intend to spend any of it. But attempts to make simple acts of technological multiplication illegal get into murky waters involving privacy, intentionality, and the tradition that what you do in your own home is your own business. The problem is as much political as it is technical, and politics, generally speaking, is not my beat. Still, there's enough engineering involved to make it worth thinking about in an engineering ethics blog.

This blog itself is an example of how nearly-free multiplication costs are used: I don’t pay to write it (except with my time and effort) and you don’t pay to read it. Still, I hope you get more than your money’s worth.

Sources: An article describing the Jammie Thomas case is at the Australian Broadcasting Corporation’s website at Lawrence Lessig’s webpage is at And an interesting comparison between copyright law and the way magicians safeguard the secrets of their tricks appears in Tim Harford’s blog

Monday, October 08, 2007

Losing By A Whisker: Lead-Free Solder and the Tin Whisker Problem

In 1998, the $250 million Galaxy IV geostationary communications satellite carrying millions of pager signals as well as the broadcast feeds of the CBS and NPR networks failed after only five years of service. Pager service wasn't restored for days and the company operating the satellite suffered considerable financial losses. Engineers determined that the problem was tiny tin whiskers that sprouted from soldered connections in the satellite's primary control processor. Because of a decision made by the European Union to prohibit the use of lead-based solder in electronics, we may see a lot more failures due to tin whiskers in the near future. How did the simple act of choosing electronic components become a complex moral issue? First, you need to understand something about tin whiskers.

When metals such as tin, zinc, and cadmium are under some kind of mechanical stress, one way they tend to relieve this stress is by sprouting tiny threads or sticks of metal called whiskers. They are very thin, much thinner than a human hair, and grow slowly over a period of months or years to a length of a few millimeters. But in the microminiature world of modern electronics, that distance is more than enough to bridge the gap between two terminals that will cause an equipment failure if shorted together. That is exactly what happened to the Galaxy IV satellite in both its primary and backup processor.

The whisker problem was first identified in the late 1940s, and since then engineers have found several ways to mitigate or eliminate it. Adding lead to tin plating or solder typically cures any whisker issues. Until very recently, the standard mixture of solder (the tin/lead alloy used to connect together most electronic components by melting it around terminals to be joined) was 60% tin and 40% lead. This alloy was reasonably inexpensive, had a low melting point, and served the electronics industry well for many decades.

In 2003, the European Union enacted a policy called Reduction of Hazardous Substances (RoHS, for short). This directive said that by July 1, 2006, most electronic products made or sold within the EU could not contain more than a very small amount of lead, cadmium, mercury, and a few other hazardous chemicals. Since the EU is a large market, and it is not practical for the thousands of electronics component manufacturers around the world to maintain two separate production lines, one for RoHS and another for non-RoHS products, this created a huge amount of turmoil in the industry as companies retooled their processes to eliminate lead from their solder, interconnection wires, plating processes, coatings, connectors, and everywhere else it was used. If you look in an electronic parts catalog these days you find "RoHS-compliant" labels on many if not most products, although non-RoHS stuff is still available, including the nasty old lead-bearing solder (which I have used, incidentally, since about the age of ten with no harmful effects). In fairness to the RoHS policy, the concern is not so much that people who use the electronics products are in any immediate danger of exposure, but that both at the manufacturing end and the recycling or disposal end, the lead can cause health problems. And that is an entirely legitimate concern.

But so is the problem of multi-million-dollar systems conking out because of tiny tin whiskers. The only commonly available RoHS-compliant solder, for example, is about 96% tin and 2% silver. Silver is not cheap, and so it costs about 50% more than the lead-bearing solder. It works all right—I've used some—but there is no lead in it to prevent the tin-whisker problem. And apparently there are few if any long-term studies of this new solder formulation that tell us how likely it is that joints soldered with it will need a shave in a few years.

The RoHS directive does exempt certain high-reliability systems such as medical devices from the no-lead requirements. But as some industry spokesmen point out, this is an empty gesture, because pretty soon it will be very hard to find any non-RoHS parts, for the simple reason that the market for them will dry up. NASA, for example, has good reason to be very concerned about the tin-whisker problem, since their satellites, and above all the Space Shuttle, contain electronic systems that are old enough to vote. So far, no life-threatening failure has occurred in the Shuttle due to tin whiskers, but the Shuttle has to keep going another two or three years at least before its commercial replacement may be available.

So what's an engineer to do? Well, the law is the law, and if your company makes or sells anything in the EU, it better comply with RoHS. As for systems that demand high reliability, there are ways around the whisker problem even if you have to use lead-free solder: wax or other impermeable coatings, proper spacing and insulating layers of other kinds, and so on. But many of these techniques are either largely untried or have problems of their own. That is what engineering is all about: solving problems. And the world will be a better place when new electronic products don't carry the burden of toxic heavy metals that they did in the past. But engineers now have to consider a new technical problem introduced by the well-meant, but perhaps technologically immature, RoHS directive. And we'll all be dealing with the consequences, perhaps in unexpected ways.

Sources: The Oct. 8, 2007 Austin American-Statesman carried an AP article by Jordan Robertson on how the high-tech industry is dealing with the challenges of tin whiskers and RoHS. Wikipedia's article "Whiskers (metallurgy)" gives a good description of the phenomenon and problems it can cause. The NASA Tin Whisker Homepage contains several pictures of actual whiskers and articles and presentations about the problem.

Monday, October 01, 2007

Battle of the Airways: How to Fix the FAA

Ladies and gentlemen! Your attention please! The Battle of the Airways is about to begin!

In this corner, we have The System. Hailed as a marvel of modern engineering when he debuted in the 1960s, The System has seen better days. Last week (Sept. 25, to be exact), he suffered a defeat at the hands of a failure in a telephone switch in Memphis, Tennessee. The scene was fantastic: air traffic controllers desperately punching numbers into their personal cellphones to call their cohorts in adjacent airspace control centers, because their radios went out and a good number of radar screens went blank, too. All flights were grounded within a 250-mile radius of Memphis, and it took the rest of the day for air traffic on the Eastern Seaboard to get back to what we call normal these days.

In this corner, we have ATA, the Air Transport Association. This airline trade association is ready to come out swinging, because they pay nearly all the taxes and fees that go to support The System. But a one-engine plane flying from Astabula, Ohio to a landing strip in an Iowa corn field takes as much or more resources from The System as a 747 pilot carrying over a hundred passengers, while paying hardly anything compared to the commercial flight.

In this corner, we have NATCA, the National Air Traffic Controllers Association. They're ready to punch somebody out before it's too late, because they've slimmed down way below weight—they've lost 10% of their numbers since 9/11/01, but air traffic's increased since then. NATCA, like The System its members operate, is getting older, smaller, and more poorly paid every day, if you believe what it tells you. And why would a fighter lie about a thing like that?

And last but not least, in this corner, we have John Q. Flying Public. Bigger than ever (individually and collectively), he's not happy about sitting in planes for hours on end and having flights canceled. Something's not right, he's pretty sure of that, but he doesn't even know who to go beat up on to fix the problem.

Waiting in the wings are the referees and the bookmakers: POTUS and Congress making the rules, and politicians and lobbyists betting on the outcome (metaphorically, we hope). The once-a-decade renewal of the FAA funding law that expired on Sept. 30, 2007 is a great opportunity for all the fighters to show their stuff. The only question is, who'll be the last man standing?

. . . Fighting is not a generally recognized way to solve complex technical disputes, but it looks like that may be how the FAA gets fixed—or doesn't, as the case may be. It may not have been a coincidence that in one week, we had a serious communications breakdown in the Memphis regional air traffic center, a Presidential statement about how the airlines had better get their act together or else, and the expiration of the current funding system for the Federal Aviation Administration, or FAA.

The technical problems are pretty clear. The present system was designed when the only way to track air traffic efficiently was with centralized radar systems that treated a 707 or a flock of birds the same way: a passive microwave-reflecting object. Identification, location, and tracking were all done either by hand or eventually by computer, but the ultimate channel through which information passed was the human air traffic controller.

That system worked great through the 70s and 80s, but as traffic has increased and newer technologies such as satellite-enabled global positioning systems (GPS) have become available, the old way of doing things has become increasingly cumbersome, unreliable, and even dangerous. Near-misses in the air are not an uncommon occurrence, and it was only by quick action on the part of already over-stressed air traffic controllers that the Memphis breakdown didn't result in a major tragedy.

Okay, we need to replace the system with a satellite-GPS-based automated one. Who's going to pay? Presently, most of the money that pays for the FAA's technology and staff (in good years, anyway) comes from ticket taxes, fees, and other sources which have little directly to do with the workload that each user represents. The Air Transport Association points out that the FAA is basically a utility, and like a water or electric company, most utilities should charge by the amount of services provided. But this is not what happens. As a result, the disconnect between funding sources and funding needs has given rise to a typical situation that often develops in government-provided services: lack of infrastructure investment and long-term planning.

How to fix it? Well, there's the good, sensible way—and the other way. The good, sensible way is for all parties involved—folks from all five or seven corners of our boxing ring, however many there are—to sit down, look at the system's needs for the next twenty years or so, figure out a big road map of how to get from here to there, and then find the money and resources to do it. This kind of thing happens all the time in private industry—the semiconductor industry, for example, has hewed closely to a roadmap of theirs that basically insures that Moore's "Law" keeps running year after year, and integrated circuits keep getting more and more complex. Airplanes aren't computer chips, but I'm talking about a planning process, not a technology.

That's the good way. The other way is to wait for a super-Memphis: something like the entire system freezing up and planes falling out of the sky, or flight delays all over the country that take a solid week to straighten out, or something equally as damaging to the airline industry as 9/11. It is my fond wish that something like this does not happen, and that the parties involved will get together and fix the problem the good way. But in a democracy, sometimes it takes a crisis to knock everybody's heads together enough to overcome differences and get things done.

Sources: A report on the Memphis breakdown can be found at the CNN website A report of President Bush's comments on Sept. 27 about the airline industry is at The Air Transport Association explains its view of FAA funding at, and the National Air Traffic Controllers Association explain some of their troubles at