Immortal media

Announcements of media technologies with very long lives happen regularly. In 2011 it was stone DVDs, in 2012 it was DNA, now it is a team from Southampton University and Eindhoven's 5-dimensional quartz DVDs. Note that in two presentations at the IEEE Mass Storage meeting back in April Hitachi announced that they already have similar but lower-density DVD technology. Below the fold, my reactions.

So far, announcements of very long-lived media have made no practical difference to large-scale digital preservation because:

• Media life is only one of many, many threats to which bits are subject. Fixing one sub-problem does not solve the overall problem.
• Very long-lived media are a niche market, and are thus very expensive. See Dave Anderson's presentation explaining why although it is perfectly possible to build hard disks with archival lives, the economics of doing so are infeasible.
• The research we have been doing in the economics of long-term preservation demonstrates the enormous barrier to adoption that accounting techniques pose for media that have high purchase but low running costs, such as these long-lived media.
• The big problem in digital preservation is not keeping bits safe for the long term, it is paying for keeping bits safe for the long term. So an expensive solution to a sub-problem can actually make the overall problem worse, not better.
• These long-lived media are always off-line media. In most cases, the only way to justify keeping bits for the long haul is to provide access to them (see Blue Ribbon Task Force). The access latency scholars (and general Web users) will tolerate rules out off-line media for at least one copy. As Rob Pike said "if it isn't on-line no-one cares any more".
• So at best these media can be off-line backups. But the long access latency for off-line backups has led the backup industry to switch to on-line backup with de-duplication and compression. So even in the backup space long-lived media will be a niche product.
• Off-line media need a reader. Good luck finding a reader for a niche medium a few centuries after it faded from the market - one of the points  Jeff Rothenberg got right two decades ago.

As I've said before, I believe that eventually DNA is likely to have a role in archival storage. It is very dense, very stable and it is very easy to create enormous numbers of copies, which mitigate many of the threats that media such as quartz DVDs don't. The quartz DVD press release indicates that the team don't understand the system-level problems of long-term storage:

“We are developing a very stable and safe form of portable memory using glass, which could be highly useful for organisations with big archives. At the moment companies have to back up their archives every five to ten years because hard-drive memory has a relatively short lifespan,” says Jingyu.

Companies don't "back up" their archives, they migrate them. And the reason for the migration isn't that hard drives have a "relatively short lifespan". The reason is that Kryder's Law has meant that after 4-5 years the old drives no longer justify their slot in the infrastructure against the much denser new drives in terms of $/byte. The drives are designed for a service life of only 4-5 years because the vendors know they will be replaced after that time even if they are still working perfectly. Hard drive media already have archival data retention times of 2-3 decades, but that has nothing to do with their service life.