Friday, February 17, 2012

Cloud Storage Pricing History

The original motivation for my work on the economics of long-term storage was to figure out whether it made sense to use cloud storage for systems such as LOCKSS. Last December I finally got around to looking at the history of Amazon S3's pricing, and I was surprised to see that in nearly 6 years the price for the first TB had dropped from $0.15/GB/mo to $0.14/GB/mo. Since then, Amazon has dropped the price to $0.125/GB/mo, so the average drop in price is now about 3%/yr.

I wondered whether this very slow price drop was representative of the cloud storage industry in general, so I went looking. Below the fold is what I found, and some of the implications for cloud use for long-term storage.
Based on this history, it appears that those planning to use cloud storage for long-term archiving should plan on storage prices dropping a few percent per year. or up to 10%/yr for very large amounts of data from some providers. Here's a preliminary graph from our simulator that assumes an initial cost of $0.13/GB/mo and interest rates matching the last 20 years. It shows the endowment in $K/TB needed to provide a 98% probability of not running out of money in 100 years for costs dropping 1-10%/yr. For example, if costs drop at the 3% historic rate for S3 we need $29K/TB.

Duracloud currently works with the first three providers, charging $6K/yr "subscription" plus $1K/TB/yr, or $1/GB/yr, or $0.083/GB/mo. $1K/TB/yr is less than the advertised price of any of the providers they work with, except if they are storing more than 5PB of data in S3. This seems to pose two problems:
  • First, as Michele points out, this model means that small customers' subscriptions subsidise large customers, which is unfair.
  • Second, it looks to me like the current pricing may be a loss leader, which would make a lot of business sense if they believed that cloud prices would drop quickly.
Unless cloud provider prices drop at least an order of magnitude faster in the future than they have in the past, Duracloud customers seem to be at risk of significant per-byte price increases.
at

7 comments:

  1. David.February 18, 2012 at 1:21 PM

    Consider keeping 8TB of data safe for 4 years. Lets compare what it would cost in the cloud versus doing an perfectly reasonable job yourself.

    Starting with DuraCloud's current pricing and assuming their per-TB price drops 3%/yr while the subscription stays constant, the total cost over 4 years would be $54,589.

    Starting with S3's current pricing and assuming that it continues to drop at 3%/yr, the total cost over 4 years would be $41,065.

    As an example of doing it yourself, suppose we want three geographically separate complete copies each protected against double disk failures. We can do this with three Drobo FS network file servers ($600 each at Amazon today) populated with 5 3TB Hitachi 5400RPM drives ($210 each at Newegg today). Add one spare for each Drobo to cover while failed drives are returned under warranty. That's a capital cost of $5580. Each Drobo consumes ~70W with all drives active (actually, in this application they'd be powered down much of the time). So we'd consume 1840 KWh over 4 years. At the price for Palo Alto Green power that would cost $250. Our experience with Drobos over several years is that they need almost no attention, but lets assume staff costs at $50/hr for 1hr/mo/box = $7200. The total cost over 4 years would be $13,030, or less than the first year's cost for Duracloud.

    ReplyDelete
  2. David.February 22, 2012 at 10:16 AM

    Via Slashdot, I see that Paul Venezia is also doubtful that cloud storage makes sense. Many of his criticisms aren't relevant to long-term archival storage, but this one is.

    "Don't forget that fast, reliable storage is very cheap these days. You can pick up 24TB of raw storage for less than $7,000, and even though they're SATA drives, they'll be more than sufficient for most general business purposes. If the lack of redundant controllers is a concern, buy two and leverage the real-time replication many NAS vendors are now offering. Heck, buy three and use one for backups. You're still ahead of the game and have tons of available disk space."

    ReplyDelete
  3. Molly TamarkinMarch 5, 2012 at 7:17 AM

    Very helpful analysis. Thank you!

    ReplyDelete
  4. Vick FisherMay 16, 2013 at 4:50 AM

    I wish more people would track the historical price of Amazon AWS S3. Thanks.


    How about an update, given the new pricing announced in early 2013 (about a 25% decrease, to 0.08/GB)?

    ReplyDelete
  5. David.August 22, 2014 at 3:53 PM

    I'm happy that Alex Teu at TechCrunch used this post as a source. I'm less than happy with the conclusions he drew and have posted a detailed debunking of his post.

    ReplyDelete
  6. UnknownOctober 21, 2020 at 4:12 PM

    Hi David. Excellent analysis. Would you be kind enough to provide an update on how your predictions performed as of 2020 and what changes you see on the horizon? Keep up the great content!

    ReplyDelete
  7. David.October 21, 2020 at 5:40 PM

    I'll add it to the queue. But don't hold your breath. Blogging is slow right now because being teaching aides, tech support, chefs and janitors for our grandkids virtual schooling is keeping us busier than we ever were when we had paying jobs.

    ReplyDelete