Dr. Pangloss and the Road-Maps

The renowned Dr. Pangloss takes great pleasure in studying the storage industry's road-maps with their rosy view of the future. I've frequently pointed to this 2008 Seagate road-map for disk technology, showing Perpendicular Magnetic Recording (PMR) being supplanted by Heat Assisted Magnetic Recording (HAMR) starting in about 2009. Its 5 years later and no vendor has yet shipped HAMR drives, although HAMR has been demonstrated in the lab at over a trillion bits per square inch, about a 30% improvement over the best current PMR. This illustrates that these vendor road-maps tend to err on the optimistic side. Dr. Pangloss is rubbing his hands with glee at the vendor's latest road-maps; below the fold I look at why he is so happy.

Our work on modelling the economics of long-term storage shows that the parameter with the biggest effect on the total cost is the Kryder rate, the rate at which bites per square inch increases with time. My co-author Daniel Rosenthal has shown that the increase in bits per square inch has in the past accounted for about 3/4 of the decrease in cost per byte, which in turn has the biggest impact on the total cost. So predictions of the future Kryder rate are interesting.

I have been predicting a significant slowing in the Kryder rate at least since just before the 2011 Thai floods made the prediction pessimistic. Many people believed that the floods and the resulting price spike were temporary, and that the Kryder rate would quickly return to its historic 30-40%. I set out the many reasons I disagreed here (PDF). Industry analysts started making more pessimistic forecasts last year, predicting no more than 20%.

Now we have road-maps from both of the disk vendors, and they both predict about 20% for the foreseeable future. Here is Seagate from last February, and here is WD from June. Remember, such vendor projections have a history of optimism.

Because HAMR has proven vastly more difficult and expensive to get into production than the vendors had expected, they have been driven to desperate measures to extend PMR. The desperate measure is shingling, in which the tracks are so close together that they overlap, needing heroic signal processing technology to disentangle on a read. The real problem with shingling is that the drives are no longer randomly writable, data has to be written an entire track at a time. This implies either:

• major changes to the file systems and disk drivers in every operating system,
• or expensive on-drive flash memory buffers to make the drives appear to be randomly writable at a significant increase in cost and reduction in performance.

Seagate is projecting the first shingled drives "later this year" and WD is projecting them next year. Seagate is projecting HAMR drives in 2014 and WD in 2016. This poses a dilemma. If HAMR is really only a year away, why go to the effort of changing all the software for shingling if the investment in doing so only has a year to pay off? On the other hand, if HAMR is really still 4-5 years away, changing the software may be the only way to achieve even modest Kryder rates. The more aggressive the vendor's projections about HAMR, the less viable shingling looks, the more it needs the additional hardware to cover up its flaws, and thus the less effective it is at driving down the cost per byte.