The False Promise of Digital Storage for Posterity

Now that almost every book, photograph, artwork, article, news item, story, drama, or film is published digitally, we are supposed to rejoice that the old-fashioned imperfect and corruptible analog forms of these media—paper that ages, film that deteriorates—has been superseded by the ubiquitous bit, which preserves data flawlessly—that is, until it doesn't.  A recent article in the engineering magazine IEEE Spectrum highlights the problems that Hollywood is having in simply keeping around usable digital copies of their old films.  And "old" in this sense can mean only three or four years ago. 

It's not like there isn't a standard way of preserving digital copies of motion pictures.  About twenty years ago, a consortium of companies got together and agreed on an open standard for magnetic-tape versions of movies and other large-volume digital material called "linear tape-open" or LTO.  If you've never heard of it, welcome to the club.  An LTO-7 cartridge is a plastic box about four inches (10 cm) on a side and a little less than an inch thick.  Inside is a reel of half-inch-wide (12 mm) tape about three thousand feet (960 m) long, and it can hold up to 6 terabytes (6 x 10¹² bytes) of uncompressed data.  Costing a little more than a hundred bucks, each cartridge is guaranteed to last at least 30 years—physically.

The trouble is, the same companies that came up with the LTO standard are part of the universal high-tech digital conspiracy to reinvent the world every two years.  Keeping something the same out of respect for the simple idea that permanence is a virtue is an entirely foreign concept to them.  Accordingly, over the last twenty years there have been seven generations of LTO tapes, and each one hasn't been backward-compatible for more than one or two generations. 

What this means to movie production companies that simply want to preserve their works digitally is this:  every three or four years at the outside, they have to copy everything they've got onto the new generation of LTO tapes.  And these tapes don't run very fast—it's not like burning a new flash drive.  Transferring an entire archive can take months and cost millions of dollars, but the customers are at the mercy of the LTO standard that keeps changing. 

According to the Spectrum article, Warner Brothers Studios has turned over the job of preserving their films to specialist film archivists at the University of Southern California, which already had a well-funded operation to preserve video interviews with Holocaust victims.  But USC faces the same digital-obsolescence issues that the studios are dealing with, and one USC archivist calls LTO tapes "archive heroin"—it's a thrill compared to the old analog archive methods, but it gets to be an expensive habit after a while.

And that gets us to a more fundamental question:  given limited resources, what should each generation preserve, in terms of intellectual output, for the next one?  And how should preservation happen?

For most of recorded history, preservation of old documents was left mostly to chance.  Now and then a forward-looking monarch would establish a library, such as the famous one in Alexandria that was established by Ptolemy I Soter, the successor of Alexander the Great, about 300 B. C.   It held anywhere from 40,000 to 400,000 scrolls, and lasted until the Romans conquered Egypt around 30 B. C., when it suffered the first of a series of fires that destroyed most of its contents. 

One can argue that the entire course of Western history would be different if all the works of the Greek philosopher Aristotle (384 B. C. - 322 B. C.) had been lost.  The way we came to possess what works we have of his is hair-raising.  After Aristotle died, his successor Theophrastus at the school where Aristotle taught, the Lyceum, inherited from Aristotle a large set of what we would call today lecture notes.   After Theophrastus died, he left them to Neleus of Scepsis, who took them from Athens, where the Lyceum was, back home to Scepsis, and stuck them in his cellar.  Then he died.  Evidently the Greek families held on to real estate back then, and it's a good thing too, because it wasn't till about 100 B. C., more than two centuries after Aristotle's passing, that Neleus's descendants had a garage sale or something, and a fellow named Apellicon of Teos found the manuscripts and bought them.  He took them back to Athens, where Apellicon's library was confiscated by the conquering Romans in 86 B. C.  Finally, some Roman philosophers realized what they had in Aristotle's works and started making copies of them around 60 B. C.

I won't even go into how most of Aristotle's works were lost again to everyone except Arabic scholars up to about 1200 A. D., but we've had enough ancient history for one blog.  The point is that historic preservation was left largely to chance until people began to realize the value of the past to the present in an organized way. 

While the movie industry deserves credit for laying out lots of money to preserve chunks of our visual cultural history, one must admit that their interests are mostly financial.  Once the people who see a movie when they're in their twenties die out, the only folks interested in such films are the occasional oddball historian or fans of specialty outlets such as the Turner Classic Films channel. 

The real problem with digital archives is not so much the fact that the technology advances so fast, although that could be alleviated.  It's the question that never has an answer until it's sometimes too late:  what is worth preserving? 

If you're a well-heeled library like the one at Harvard University, the answer is simple:  everything you get your hands on.  But most places are not that well off, so it's a judgment call as to what to toss and what to keep using the always-limited resources at hand.

Despite the best intentions of well-funded film archivists, my suspicion is that a few centuries hence, we will find that many of the works of most importance to the future, whatever they are, were preserved not on purpose, but by hair-raising combinations of fortunate accidents like the ones that brought us the works of Aristotle.  And if I'm wrong, well, chances are this blog won't be one of those things that are preserved.  So nobody will know.

Sources:  The article "The Lost Picture Show:  Hollywood Archivists Can't Outpace Obsolescence" by Marty Perlmutter appeared in the May 2017 issue of IEEE Spectrum and online at http://spectrum.ieee.org/computing/it/the-lost-picture-show-hollywood-archivists-cant-outpace-obsolescence?.  The story of how Aristotle's works came down to us is reported independently by at least two ancient sources, and so is probably pretty close to the truth, according to the Wikipedia article on Aristotle.  I also referred to Wikipedia articles on the Library of Alexandria and the Ptolemaic dynasty.