Library of Congress Storage Architecture Meeting 2026

Once again I attended most of the library of Congress' Designing Storage Architectures workshop remotely. I apologize for the delay in posting this; domestic duties have kept me very busy recently. Below the fold notes on the talks that caught my attention, based on my now somewhat memory and the slide decks for the talks from the Library of Congress website.

Compute In Storage

Tobias Mann's Los Alamos Taps Seagate To Put Compute On Spinning Rust describes progress in the concept of computational storage. I first discussed this in my 2010 JCDL keynote, based on 2009's FAWN, the Fast Array of Wimpy Nodes by David Anderson et al from Carnegie-Mellon. Their work started from the observation that moving data from storage to memory for processing by a CPU takes time and energy, and the faster you do it the more energy it takes. So the less of it you do, the better. Below the fold I start from FAWN and end up with the work under way at Los Alamos.

The Medium-Term Prospects for Long-Term Storage Systems

Back in May I posted The Future of Storage, a brief talk written for a DARPA workshop of the same name. The participants were experts in one or another area of storage technology, so the talk left out a lot of background that a more general audience would have needed. Below the fold, I try to cover the same ground but with this background included, which makes for a long post.

This is an enhanced version of a journal article that has been accepted for publication in Library Hi Tech, with images that didn't meet the journal's criteria, and additional material reflecting developments since submission. Storage technology evolution can't be slowed down to the pace of peer review.

The Future of Storage

My preparation for a workshop on the future of storage included giving a talk at Seagate and talking to the all-flash advocates. Below the fold I attempt to organize into a coherent whole the results of these discussions and content from a lot of earlier posts.

Talk at Seagate

I gave a talk at Seagate as part of a meeting to prepare myself for an upcoming workshop on The Future of Storage. It pulls together ideas from many previous posts. Below the fold, a text of the talk with links to the sources that has been edited to reflect some of what I learnt from the discussions.

The Dawn of DAWN?

At the 2009 SOSP David Anderson and co-authors from C-MU presented FAWN, the Fast Array of Wimpy Nodes. It inspired me to suggest, in my 2010 JCDL keynote, that the cost savings FAWN realized without performance penalty by distributing computation across a very large number of very low-power nodes might also apply to storage.

The following year Ian Adams and Ethan Miller of UC Santa Cruz's Storage Systems Research Center and I looked at this possibility more closely in a Technical Report entitled Using Storage Class Memory for Archives with DAWN, a Durable Array of Wimpy Nodes. We showed that it was indeed plausible that, even at then current flash prices, the total cost of ownership over the long term of a storage system built from very low-power system-on-chip technology and flash memory would be competitive with disk while providing high performance and enabling self-healing.

Although flash remains more expensive than hard disk, since 2011 the gap has narrowed from a factor of about 12 to about 6. Pure Storage recently announced FlashBlade, an object storage fabric composed of large numbers of blades, each equipped with:

• Compute – 8-core Xeon system-on-a-chip – and Elastic Fabric Connector for external, off-blade, 40GbitE networking,
• Storage – NAND storage with 8TB or 52TB raw capacity of raw capacity and on-board NV-RAM with a super-capacitor-backed write buffer plus a pair of ARM CPU cores and an FPGA,
• On-blade networking – PCIe card to link compute and storage cards via a proprietary protocol.

Chris Mellor at The Register has details and two commentaries.

FlashBlade clearly isn't DAWN. Each blade is much bigger, much more powerful and much more expensive than a DAWN node. No-one could call a node with an 8-core Xeon, 2 ARMs, and 52TB of flash "wimpy", and it'll clearly be too expensive for long-term bulk storage. But it is a big step in the direction of the DAWN architecture.

DAWN exploits two separate sets of synergies:

• Like FlashBlade, it moves the computation to where the data is, rather then moving the data to where the computation is, reducing both latency and power consumption. The further data moves on wires from the storage medium, the more power and time it takes. This is why Berkeley's Aspire project's architecture is based on optical interconnect technology, which when it becomes mainstream will be both faster and lower-power than wires. In the meantime, we have to use wires.
• Unlike FlashBlade, it divides the object storage fabric into a much larger number of much smaller nodes, implemented using the very low-power ARM chips used in cellphones. Because the power a CPU needs tends to grow faster than linearly with performance, the additional parallelism provides comparable performance at lower power.

So FlashBlade currently exploits only one of the two sets of synergies. But once Pure Storage has deployed this architecture in its current relatively high-cost and high-power technology, re-implementing it in lower-cost, lower-power technology should be easy and non-disruptive. They have done the harder of the two parts.

SSDs: Cheap as Chips?

Four and a half years ago Ian Adams, Ethan Miller and I proposed DAWN, a Durable Array of Wimpy Nodes, pointing out that a suitable system design could exploit the characteristics of solid-state storage to make system costs for archival storage competitive with hard disk despite greater media costs. Since then, the cost differential between flash and hard disks has decreased substantially. Below the fold, an update.

EE380 talk on eBay storage

Russ McElroy & Farid Yavari gave a talk to Stanford's EE380 course describing how eBay's approach to storage (YouTube) is driven by their Total Cost of Ownership (TCO) model. As shown in this screengrab, by taking into account all the cost elements, they can justify the higher capital cost of flash media in much the way, but with much more realistic data and across a broader span of applications, that Ian Adams, Ethan Miller and I did in our 2011 paper Using Storage Class Memory for Archives with DAWN, a Durable Array of Wimpy Nodes.

We were inspired by a 2009 paper FAWN A Fast Array of Wimpy Nodes in which David Andersen and his co-authors from C-MU showed that a network of large numbers of small CPUs coupled with modest amounts of flash memory could process key-value queries at the same speed as the networks of beefy servers used by, for example, Google, but using 2 orders of magnitude less power.

As this McElroy slide shows, power cost is important and it varies over a 3x range (a problem for Kaminska's thesis about the importance of 21 Inc's bitcoin mining hardware). He specifically mentions the need to get the computation close to the data, with ARM processors in the storage fabric. In this way the amount of data to be moved can be significantly reduced, and thus the capital cost, since as he reports the cost of the network hardware is 25% of the cost of the rack, and it burns a lot of power.

At present, eBay relies on tiering, moving data to less expensive storage such as consumer hard drives when it hasn't been accessed in some time. As I wrote last year:

Fundamentally, tiering like most storage architectures suffers from the idea that in order to do anything with data you need to move it from the storage medium to some compute engine. Thus an obsession with I/O bandwidth rather than what the application really wants, which is query processing rate. By moving computation to the data on the storage medium, rather than moving data to the computation, architectures like DAWN and Seagate's and WD's Ethernet-connected hard disks show how to avoid the need to tier and thus the need to be right in your predictions about how users will access the data.

That post was in part about Facebook's use of tiering, which works well because Facebook has highly predictable data access patterns. McElroy's talk suggests that eBay's data accesses are somewhat predictable, but much less so than Facebook's. This makes his implication that tiering isn't a good long-term approach plausible.

Five Minutes Of Fame

On Monday, Chris Mellor at The Register had a piece with a somewhat misleading title that provides a good summary of the argument we've been making since at least early 2011 that the Kryder rate, the rate of annual decrease in the cost per byte of storage, had slowed dramatically. As we have shown, this slowing has huge implications for the cost of long-term storage.

Today, Chris is back with a similar summary of Preeti Gupta et al's MASCOTS paper, An Economic Perspective of Disk vs. Flash Media in Archival Storage. This paper reports on some more sophisticated economic modelling that supports the argument of DAWN: a Durable Array of Wimpy Nodes. This 2011 technical report showed that, using a similar fabric to Carnegie-Mellon's 2009 FAWN: a Fast Array of Wimpy Nodes for long-term storage instead of computation, the running costs would be low enough to overcome the much higher cost of the flash media as compared to disk

A Challenge to the Storage Industry

I gave a brief talk at the Library of Congress Storage Architecture meeting, pulling together themes from a number of recent blog posts. My goal was twofold:

• to outline the way in which current storage architectures fail to meet the needs of long-term archives,
• and to set out what an architecture that would meet those needs would look like.

Below the fold is an edited text with links to the earlier posts here that I was condensing.

Is This The Dawn of DAWN?

More than three years ago, Ian Adams, Ethan Miller and I were inspired by a 2009 paper FAWN: A Fast Array of Wimpy Nodes from David Andersen et al at C-MU. They showed how a fabric of nodes, each with a small amount of flash memory and a very low-power processor, could process key-value queries as fast as a network of beefy servers using two orders of magnitude less power.

We put forward a storage architecture called DAWN: Durable Array of Wimpy Nodes, similar hardware but optimized for long-term storage. Its advantages were small form factor, durability, and very low running costs. We argued that these would outweigh the price premium for flash over disk. Recent developments are starting to make us look prophetic - details below the fold.

Talk at Seagate

I gave a talk at Seagate entitled:

Storage Will Be
Much Less Free
Than It Used To Be

Below the fold is an edited text with links to the sources.

What Could Possibly Go Wrong?

I gave a talk at UC Berkeley's Swarm Lab entitled "What Could Possibly Go Wrong?" It was an initial attempt to summarize for non-preservationistas what we have learnt so far about the problem of preserving digital information for the long term in the more than 15 years of the LOCKSS Program. Follow me below the fold for an edited text with links to the sources.

Implementing DAWN?

In a 2009 paper "FAWN A Fast Array of Wimpy Nodes" David Andersen and his co-authors from C-MU showed that a network of large numbers of small CPUs coupled with modest amounts of flash memory could process key-value queries at the same speed as the networks of beefy servers used by, for example, Google, but using 2 orders of magnitude less power. In 2011, Ian Adams, Ethan Miller and I proposed extending this concept to long-term storage in a paper called “Using Storage Class Memory for Archives with DAWN, a Durable Array of Wimpy Nodes”. DAWN was just a concept, we never built a system.

Now, in a fascinating talk at the Chaos Computer Conference called "On Hacking MicroSD Cards" the amazing Bunnie Huang and his colleague xobs revealed that much of the hardware for a DAWN system may already be on the shelves at your local computer store. Below the fold, details of the double-edged sword that is extremely low-cost hardware, to encourage you to read the whole post and watch the video of their talk.

Seagate's Ethernet Hard Drives

A week ago Seagate made an extraordinarily interesting announcement for long-term storage, their Kinetic Open Storage Platform, including disk drives with Ethernet connectivity. Below the fold, the details.

Talk at Digital Preservation 2013

I was on a panel at the Library of Congress' Digital Preservation 2013 meeting entitled Green Bytes: Sustainable Approaches to Digital Stewardship. Below the fold is the text of my brief presentation, with links to the sources.

DAWN vs. Twitter

I blogged three weeks ago about the Library of Congress ingesting the Twitter feed, noting that the tweets were ending up on tape. It is over 130TB and growing 190GB/day. The Library is still trying to work out how to provide access to this collection; for example they cannot afford the infrastructure that would allow readers to perform keyword searches. This leaves the 400-odd researchers who have already expressed a need for access to the collection stymied. The British Library is also running into problems providing access to large collections, although not as large as Twitter. They are reduced to delivering 30TB NAS boxes to researchers, the same approach as Amazon and other services have taken to moving large amounts of data.

I mentioned this problem in passing in my earlier post, but I have come to understand that this observation has major implications for the future of digital preservation. Follow me below the fold as I discuss them.

Modeling the Economics of Long-Term Storage

I gave a talk at the Library of Congress workshop on Designing Storage Architectures entitled Modeling the Economics of Long-Term Storage. It was talk about work in progress with Library of Congress funding, expanding ideas I described in these two blog posts about ways to compare the costs of different approaches to long term storage. I had only 10 minutes to speak, so below the fold is an expanded and edited text of the talk with links to the sources.

Solid State Memory for Archival Use

In last year's JCDL keynote I pointed to work at Carnegie-Mellon on FAWN, the Fast Array of Wimpy Nodes and suggested that the cost savings FAWN realizes by distributing computation across a very large number of very low-power nodes might also apply to storage. Now, Ian Adams and Ethan Miller of UC Santa Cruz's Storage Systems Research Center and I have looked at this possibility more closely in a Technical Report entitled Using Storage Class Memory for Archives with DAWN, a Durable Array of Wimpy Nodes. We show that it is indeed plausible that, even at current flash memory prices, the total cost of ownership over the long term of a storage system built from very low-power system-on-chip technology and flash memory would be competitive with disk. More on this below the fold.