Thursday, March 1, 2018

Archival Media: Not a Good Business

Thinking more about DNA's Niche in the Storage Market led me to focus on some problems with the market for archival media in general, not just DNA. The details are below the fold.

Archival-only media such as steel tape, silica DVDs, 5D quartz DVDs, and now DNA face some fundamental business model problems because they function only at the very bottom of the storage hierarchy. The usual diagram of the storage hierarchy, like this one from the Microsoft/UW team researching DNA storage, makes it look like the size of the market increases downwards. But that's very far from the case.

2016 Media Shipments

Exabytes Revenue $/GB
Hard Disk693$26.8B$0.039
LTO Tape40$0.65B$0.016
This table, from Robert Fontana and Gary Decad's Moore’s law realities for recording systems and memory storage components: HDD, tape, NAND, and optical, shows that the size of the market in dollar terms decreases downwards. LTO tape is less than 1% of the media market in dollar terms and less than 5% in capacity terms. Archival media are a very small part of the storage market.

Why is this? The upper layers of the hierarchy generate revenue; the archival layer is purely a cost. If the data are still generating revenue, at least one copy is on flash or hard disk. Even if there is a copy in the archive, that one isn't generating revenue. Facebook expects the typical reason for a read request  for data from their Blu-Ray cold storage will be a subpoena. Important, but not a revenue generator. So archival media are a market where customers are reluctant to spend, because there is no return on the investment.

This means that both revenue and margins decrease down the hierarchy, and thus that R&D spending decreases down the hierarchy. R&D spending on a new archival medium is aimed at a market with low revenues and low margins. Not a good investment decision.

But that isn't the worst prospect facing a new archival medium. As we currently see with flash, R&D investment in storage media is focused at the top of the hierarchy, where the revenues and margins are best. The result is to push legacy media, currently hard disk, down the hierarchy. Thus new, archival-only media have to compete with legacy universal media being pushed down the stack. They face two major disadvantages:
  • The legacy medium's investment in R&D and manufacturing capacity has been amortized at the higher levels of the hierarchy, whereas the new medium's R&D and manufacturing investments have to earn their whole return at the archival layer. So the legacy medium is likely to be cheaper.
  • The legacy medium has latency and bandwidth suited to the higher layers of the hierarchy, albeit some time ago. It thus out-performs the new, archival-only medium.
The price/performance playing field is far from level. The market is small, with low margins. This isn't a good business to be in.


David. said...

Yet another long-lived medium company believes it can solve the problems of archival data:

"Can data survive in space over extremely long times and multiple human generations? The possibility of human colonies on other planets may ultimately depend on just such data stability. Now, a patented innovative long-term archival data storage system created by a Delaware-based firm is being tested on the International Space Station (ISS) for up to a year.

The system developed by Creative Technology LLC (CTech) of Hockessin, DE, applies a century-old tested archival media for photography in a completely new way for storing high-density computer data in perpetuity. Inherently secure, low-cost technology is used that cannot be hacked or altered. CTech’s archival media can be used to store critical DNA and healthcare records, financial information and contracts, family photos and records which need to preserved for multiple human generations"

David. said...

LTO tape media manufacturers are shooting themselves in the foot with a patent lawsuit. In LTO-8 tape media patent lawsuit cripples supply as Sony and Fujifilm face off in court Chris Mellor describes the convoluted history of the case, and concludes:

"With tape archiving currently under threat from fast access disk vaults and even faster access QLC (4bits/cell) flash down the road, the tape suppliers are doing a damn good job of strangling the tape sector."