Economics of Evil

Back in March Google announced that this weekend is the end of Google Reader, the service many bloggers and journalists used to use to read online content via RSS. This wasn't the first service Google killed, but because the people who used the service write for the Web, the announcement sparked a lively discussion. Because many people believe that commercial content platforms and storage services will preserve digital content for the long term, the discussion below the fold should be of interest here.

Paul Krugman commented:

Google’s decision to shut down Google Reader has ... provoked a lot of discussion about the future of web-based services. The most interesting discussion, I think, comes from Ryan Avent, who argues that Google has been providing crucial public infrastructure — but doesn’t seem to have an interest in maintaining that infrastructure.

The Ryan Avent post at The Economist's Free Exchange blog that Krugman linked to pointed out that dependence on Google's services has a big impact on the real world:

Once we all become comfortable with [Google's services] we quickly begin optimising the physical and digital resources around us. Encyclopaedias? Antiques. Book shelves and file cabinets? Who needs them? And once we all become comfortable with that, we begin rearranging our mental architecture. We stop memorising key data points and start learning how to ask the right questions. We begin to think differently. About lots of things. We stop keeping a mental model of the physical geography of the world around us, because why bother? We can call up an incredibly detailed and accurate map of the world, complete with satellite and street-level images, whenever we want. ... The bottom line is that the more we all participate in this world, the more we come to depend on it. The more it becomes the world.

Avent described Google's motivation for providing the infrastructure:

Google has asked us to build our lives around it: to use its e-mail system ..., its search engines, its maps, its calendars, its cloud-based apps and storage services, its video- and photo-hosting services, ... It hasn't done this because we're its customers, it's worth remembering. We aren't; we're the products Google sells to its customers, the advertisers. Google wants us to use its services in ways that provide it with interesting and valuable information, and eyeballs.

So, Google may have good reasons for killing some services:

It's a big company, but even big companies have finite resources, and devoting those precious resources to something that isn't making money and isn't judged to have much in the way of development potential is not an attractive option. ... Someone else will come along to provide the service and, if they give it their full attention, to improve it.

And indeed alternate services such as Feedly are taking over. The only push-back for Google is weak:

But that makes it increasingly difficult for Google to have success with new services. Why commit to using and coming to rely on something new if it might be yanked away at some future date?

Google only partially mitigates this through their Data Liberation Front, whose role is to ensure that users of their services can extract their data in a re-usable form. But there remains a problem for society:

That's a lot of power to put in the hands of a company that now seems interested, mostly, in identifying core mass-market services it can use to maximise its return on investment.

What this implies is that Google is becoming an essential utility:

the history of modern urbanisation is littered with examples of privately provided goods and services that became the domain of the government once everyone realised that this new life and new us couldn't work without them.

Paul Krugman described the economics underlying this:

even in a plain-vanilla market, a monopolist with high fixed costs and limited ability to price-discriminate may not be able to make a profit supplying a good even when the potential consumer gains from that good exceed the costs of production. Basically, if the monopolist tries to charge a price corresponding to the value intense users place on the good, it won’t attract enough low-intensity users to cover its fixed costs; if it charges a low price to bring in the low-intensity user, it fails to capture enough of the surplus of high-intensity users, and again can’t cover its fixed costs.

What Avent adds is network externalities, in which the value of the good to each individual user depends on how many others are using it. ... they mean that if the monopolist still doesn’t find it worthwhile to provide the good, the consumer losses are substantially larger than in a conventional monopoly-pricing analysis.

So what’s the answer? As Avent says, historical examples with these characteristics — like urban transport networks — have been resolved through public provision. It seems hard at this point to envision search and related functions as public utilities, but that’s arguably where the logic will eventually lead us.

It is indeed hard to envision these services as public utilities; they are world-wide rather than national or local, and the communication infrastructure on which they rely, despite being utility-like, is provided by for-profit, lightly regulated companies.

What does this mean for digital preservation?

• "Free" services, such as Google Drive, in which the user is the product rather than the customer should never be depended upon. At any moment they may suffer the fate of Google Reader and many other Google services; because the user is not the customer there is essentially no recourse.
• Over time any business model for extracting value from content will become less and less effective. There are several reasons. Competitors will arise and decrease the available margins. As content accumulates, the average age of an item will increase; it is easier to extract value from younger content. Thus the service is continually in a race between this decreasing effectiveness and the cost reduction from improving technology such as Kryder's Law for storage. Once the cost reduction fails to outpace the value reduction the service is probably doomed. So the slowing pace of storage cost reduction is likely to cause more casualties, especially among those services that accumulate content.
• Because archived content is rarely accessed it generates little valuable information and lacks network effects, making it a poor business for the provider. This casts another shadow over the future of free or all-you-can-eat storage services.
• Although the user of services such as Google Cloud Storage and S3 is a paying customer, digital preservation will be a very small part of the service's customer base. Thus the incentives for the service provider to continue or terminate those aspects of the service that make it suitable for digital preservation will not be greatly different from those of a free service.
• Because digital preservation is a relatively small market, services tailored to its needs will lack the economies of scale of the kind of infrastructure services Avent and Krugman are discussing. Thus they will be more expensive. But the temptation will always be to implement them as a thin layer of customization over generic infrastructure services, as we see with Duracloud and Preservica, to capture as much of the economies of scale as possible. This leaves them vulnerable to the whims of the infrastructure provider. See also my comment on on-demand vs. base-load economics.