Facebook's "Cold Storage"

Last week Facebook announced they are building a couple of "cold storage" data centers:

Facebook will move older pictures and back-up photos to new-more energy efficient-data centers, called "cold storage" centers.  ... the new "cold storage" centers-which are still in construction-will be five times more energy efficient and will allow users to access old images anytime without noticing any difference.

Facebook's problem is that they are ingesting 315M photos/day, or 7PB/month. Reducing the energy consumed by the backup copies and the older, less frequently accessed pictures is important. Although this is a work-in-progress and Facebook isn't talking about some details, it appears that among the techniques they are using are erasure coding, to operate with a lower replication factor, aggressively spinning down disks, using flash to hold indexes, and perhaps new, low-power drives such as these from Seagate, which claim 27% less power draw. They are working in the context of the Open Vault project, so this technology should eventually be available to others.

Kai Li's FAST Keynote

Kai Li's keynote at the FAST 2013 conference was entitled Disruptive Innovation: Data Domain Experience. Data Domain was the pioneer of deduplication for backups. I was one of the people Sutter Hill asked to look at Data Domain when they were considering a B-round investment in 2003. I was very impressed, not just with their technology, but more with the way it was packaged as an appliance so that it was very easy to sell. The elevator pitch was "It is a box. You plug it into your network. Backups work better."

I loved Kai's talk. Not just because I had a small investment in the B round, so he made me money, but more because just about everything he said matched experiences I had at Sun or nVIDIA. Below the fold I discuss some of the details.

Thoughts from FAST 2013

I attended Usenix's 2013 FAST conference. I was so interested in Kai Li's keynote entitled Disruptive Innovation: Data Domain Experience that I'll devote a separate post to it. Below the fold are some other things that caught my attention. Thanks to Usenix's open access policy, you can follow the links and read the papers if I've piqued your interest.

Amazon's margins

I've been blogging a lot about the economics of cloud storage, and always using Amazon as the comparison. I've been stressing that the margins on their cloud storage business are extortionate. But Amazon is famous for running on very low margins. Below the fold I look at this conundrum.

Rothenberg still wrong

Last March Jeff Rothenberg gave a keynote entitled Digital Preservation in Perspective:How far have we come, and what's next? to the Future Perfect 2012 conference at the wonderful, must-visit Te Papa Tongarewa museum in Wellington, New Zealand. The video is here. The talk only recently came to my attention, for which I apologize.

I have long argued, for example in my 2009 CNI keynote, that while Jeff correctly diagnosed the problems of digital preservation in the pre-Web era, the transition to the Web that started in the mid-90s made those problems largely irrelevant. Jeff's presentation is frustrating, in that it shows how little his thinking has evolved to grapple with the most significant problems facing digital preservation today. Below the fold is my critique of Jeff's keynote.