Tuesday, February 26, 2013

Facebook's "Cold Storage"

Last week Facebook announced they are building a couple of "cold storage" data centers:
Facebook will move older pictures and back-up photos to new-more energy efficient-data centers, called "cold storage" centers.  ... the new "cold storage" centers-which are still in construction-will be five times more energy efficient and will allow users to access old images anytime without noticing any difference.
Facebook's problem is that they are ingesting 315M photos/day, or 7PB/month. Reducing the energy consumed by the backup copies and the older, less frequently accessed pictures is important. Although this is a work-in-progress and Facebook isn't talking about some details, it appears that among the techniques they are using are erasure coding, to operate with a lower replication factor, aggressively spinning down disks, using flash to hold indexes, and perhaps new, low-power drives such as these from Seagate, which claim 27% less power draw. They are working in the context of the Open Vault project, so this technology should eventually be available to others.

No comments: