Future's So Bright, We Gotta Wear Shades

In Sun Microsystems' first decade the company was growing exponentially and one of Scott McNealy's favorite slogans was "The Future's So Bright, We Gotta Wear Shades", adapted from a song by Timbuk 3. McNealy missed the ironic intent of the song, but the slogan was appropriate. Exponential growth continued most of the way through the company's second decade. Eventually, however, growth stopped.

It is always easy to believe that exponential growth will continue indefinitely, but it won't. It can't. There are always resource or other constraints that, at best, cause it to flatten. More likely, the growth will overshoot and collapse, as Sun's did. This problem is common to all exponential growth; a wonderful illustration is this post by Prof. Tom Murphy of UCSD:

U.S. energy use since 1650 shows a remarkably steady growth trajectory, characterized by an annual growth rate of 2.9%

...

For a matter of convenience, we lower the energy growth rate from 2.9% to 2.3% per year so that we see a factor of ten increase every 100 years. We start the clock today, with a global rate of energy use of 12 terawatts (meaning that the average world citizen has a 2,000 W share of the total pie).

...

No matter what the technology, a sustained 2.3% energy growth rate would require us to produce as much energy as the entire sun within 1400 years. A word of warning: that power plant is going to run a little warm. Thermodynamics require that if we generated sun-comparable power on Earth, the surface of the Earth—being smaller than that of the sun—would have to be hotter than the surface of the sun!

Views of the future for digital preservation are formed by two exponential growth projections; Moore's Law, projecting that density of transistors on a chip will increase exponentially, and Kryder's Law, projecting that the density of bits in storage will increase exponentially. Thus we plan on the basis that future costs of processing and storage will be much less than current costs.

An accessible explanation of the troubles Moore's Law is encountering is in Rik Myslewski's report for The Register on Simon Segars' keynote at the Hot Chips conference. Segars is employee #16 and head of the Physical IP division at ARM.

"Silicon scaling has been great." Segars reminisces. "We've gotten huge gains in power, performance, and area, but it's going to end somewhere, and that's going to affect how we do design and how we run our businesses, so my advice to you is get ready for that.

"It's coming sooner than a lot of people want to recognize."

I've been warning that we're likely to fall behind the Kryder's Law curve for some time, because the transition to the next disk drive technology is proving harder than expected and because the advantages of solid state storage come with higher costs. One thing I failed to anticipate is:

Multiple manufacturers in the IT industry have been keeping a wary eye on China's decision to cut back on rare earth exports and the impact it may have on component prices. An article from DigiTimes suggests consumers will see that decision hit the hard drive industry this year, with HDD prices trending upwards an estimated 5-10 percent depending on capacity.

This isn't likely to be as significant a factor as the others, but it is yet another straw in the wind.

Moonalice plays Palo Alto

On July 23^rd the band Moonalice played in Rinconada Park as part of Palo Alto's Twilight Concert series. The event was live-streamed over the Internet. Why is this interesting? Because Moonalice streams and archives their gigs in HTML5. Moonalice is Roger McNamee's band. Roger is a long-time successful VC (he invested in Facebook and Yelp, for example) and, based on his experience using HTML5 for the band, believes that HTML5 will transform the Web. On June 28^th he spoke at the Paley Center for the Media and explained why. An overview appeared at Business Insider, but you really need to watch the full video of the talk. Watch it then follow me below the fold. I'll explain some implications for digital preservation.

A Brief History of E-Journal Preservation

The workshop on the Future of Research Communication opened with a set of talks about the past, providing background for the workshop discussions. My talk covered the history of e-journal preservation, and the lessons that can be drawn from it. An edited text of the talk, with links to the sources, is below the fold.

Rebuttals

I'll save those who might point to evidence against some of my positions the trouble by pointing to the evidence myself:

• I've used the comparison between libraries and Starbucks several times. According to Jim Romenesko via Gawker and Yves Smith we learn that Starbucks is succeeding so well that the students and other "laptop hobos" are driving out people who want to drink coffee. So maybe I was wrong and libraries are showing how hard it is to compete with free.
• On the other hand, I have been very skeptical of the New York Times' paywall. In the Columbia Journalism Review Felix Salmon points to Seth Mnookin reporting on the first 4 months of the paywall.

  its digital-subscription plan has thus far been an enormous success. The ... the goal was to amass 300,000 online subscribers within a year of launch. On Thursday, the company announced that after just four months, 224,000 users were paying for access to the paper’s website. Combined with the 57,000 Kindle and Nook readers who were paying for subscriptions and the roughly 100,000 users whose digital access was sponsored by Ford’s Lincoln division, that meant the paper had monetized close to 400,000 online users. (Another 756,000 print subscribers have registered their accounts on the Times’ website.)

  Clearly that is encouraging, and Salmon goes on to point out that:

  Those paying digital subscribers, however, are much more valuable than their subscription streams alone would suggest. They’re hugely loyal, they read loads of stories, they’re well-heeled, and advertisers will pay a premium to reach them. ... it seems as though total digital ad revenues are going up, not down, as subscriptions get introduced: the holy grail of paywalls.

  However, this is only one digital subscriber for every two paper subscribers, and the NYT is making much less from each digital subscriber even after advertising revenue, so I'm sticking to my position.
• I have been skeptical of the long-term prospects of large publishers such as Elsevier, although I was careful to note my track record of being right way too soon. In the short term, Elsevier's parent Reed Elsevier recently turned in modestly better resultsfor the first half of 2011. Its stock has been performing roughly in line with the S&P500. I still believe that the scope for Elsevier to acquire large numbers of lucrative new customers is limited, and their ability to squeeze more income from the existing customers is obviously limited by research and teaching budgets.

Fujitsu agrees with me

I've been saying for some time that flash memory is unlikely to be the long-term solid state memory technology. In an interview with Chris Mellor at The Register, Fujitsu's CTO agrees with me:

[Joseph Reger] reckons Phase Change Memory (PCM) is the closest, in terms of time to become a usable technology, than other post-flash contenders such as HP's Memristor.

In the more interesting part of the story, he agrees with me that the real potential of these post-flash technologies is that they can be packaged as persistent RAM rather than block storage:

[Reger] asks if everything will be rewritten and re-orchestrated to work with data memory management. Is there effectively only going to be one tier, memory in one form or another?

"Currently, having data in storage means it's not in memory. Is it going to stay like that?" After all, storage was invented to deal with memory-size limitations. If those limitations go away then who needs storage?

Reger said: "I truly believe we are going to have a data orientation rather than memory and storage orientations." But this is really far out in the future.

I'm old enough to remember when computer memory persisted across power cycles because it was magnetic cores. I'd love to see this feature return. The major software change that would be needed is far more than simply using in-memory databases. The RAM data structures would need to be enhanced with metadata and backups, especially for long-term integrity, if we were to get rid of block storage entirely.