- The question arose during discussions at the Dutch National Library in September. Here is an interesting blog post in which some potential examples of such format obsolescence are suggested and refuted.
- In the Q&A for a session at iPRES 2010 I asked whether anyone in the audience had been forced to migrate a widely-used format to retain legibility, as opposed to choosing to do so. No-one had.
- The first response to my November CACM paper on bit preservation raised the issue of format obsolescence. It and my response
are due toappear in the DecemberJanuary letters to the editor of CACM.
Suppose, for the purpose of making the discussion easy, that formats have lifetimes that are randomly distributed about a mean lifetime. This mean lifetime would represent that half-life of digital formats; I applied the same concept to the half-life of bits in this post. The half-life of digital formats is an important number for digital preservation, worthy of serious research. In the meantime, can we estimate what this half-life of formats is?
The fact that we have not observed an instance of format obsolescence in the last 15 years allows us to make a rough estimate. Assume that there are 100 widely used formats (probably an underestimate). Then if the half-life were 100 years we should have seen 7 formats go obsolete. We saw none, so if we accept these assumptions we should be fairly confident that the half-life of a widely used digital format is more than 100 years.
In facing the prospect that a format will eventually go obsolete, we have a choice between spending money now preparing for it, or waiting until the format goes obsolete and spending money recovering. The longer the half-life, the better waiting looks. Lets look at the extreme positions first:
- By spending $X now, we ensure that the cost when the format goes obsolete is $0.
- By spending $0 now, we ensure that the cost when the format goes obsolete is $Y.
If the half-life is 100 years, and the long-term real interest rate is 3%, and the eventual cost (in current dollars) of dealing with obsolescence is $1, this means that the most we can afford to spend right now preparing for it is about $0.05. And we can only afford to spend that much now if we are sure that by doing so we guarantee that the eventual cost of obsolescence is $0.
We have seen no obsolescence in the past 15 years. Assume that half of all formats will go obsolete in the next 15 years; even the proponents of format migration would admit that this is extraordinarily unlikely. Even in this extreme scenario, it is better to spend $1.56 when the format goes obsolete than to spend $1 now. And, if you do spend $1 now, for every cent you end up spending when obsolescence happens, the $1.56 is increased by one cent.
Another way of looking at this is to realize that we can never be sure that whatever we do now to prepare for format obsolescence will work. Suppose there is a 20% chance that spending $1 now doesn't help, and we end up spending the $1.56 as well. We would be better off spending nothing now and spending $1.87 when obsolescence happens.
What this analysis shows is that even in exceptionally pessimistic scenarios to justify spending $1 now on preparing for format obsolescence we have to be sure that doing so is more effective than spending about double that when obsolescence happens. In scenarios that conform more closely to what we observe in the real world, we would have to be sure that spending $1 now is more effective than spending about $20 when it is needed.
Links to my earlier posts on this topic: