Wednesday, January 21, 2015

New Yorker on Web Archiving

Do not hesitate, do not pass Go, right now please read Jill Lepore's really excellent New Yorker article Cobweb: can the Web be archived?

Tuesday, January 20, 2015

A Solution To Everything?

Twenty-five days ago I wrote a critique of the idea that blockchain based crypto-currencies could be a basis for peer-to-peer storage. My main argument was from an earlier post, that economies of scale meant a successful currency would not in fact be decentralized. But the occasion for the post was the observation that Bitcoin's value was collapsing, reinforcing the economies of scale argument by reducing the income of miners, and thus forcing all but the lowest-cost (largest) miners to drop out, as shown by the hash rate ceasing to grow.

Since then, there has been a flood of proposals to base other P2P storage systems, election voting, even a replacement for the Internet on blockchain technology. Every one of these proposals for using the blockchain as a Solution for Everything I've looked at appears to make three highly questionable assumptions:
About a year ago Bitcoin peaked at over $1100. When I wrote that post a Bitcoin was worth $319. Today it is worth $215. It has lost one-third of its value in less than a month, and over 80% in a year. In less than a month miner's revenue has gone from about $1.4M to less than $800K, a drop of 43%. Below the fold, more on the implications of this price implosion.

Tuesday, January 13, 2015

The Miner's Dilemma

I've pointed out how economies of scale lead to concentration of mining power in block-chain based peer-to-peer (P2P) systems such as Bitcoin, which impairs the decentralized nature of such systems, their major selling point. As Ittay Eyal points out in an important post entitled The Miner's Dilemma:
The dismantling of overly large pools is one of the most important and difficult tasks facing the Bitcoin community.
Pools are needed to generate consistent income but:
[Miners] can get steady income from pools well below 10%, and they have only little incentive to use very large pools; it's mostly convenience and a feeling of trust in large entities.
As I write, the three largest pools (F2Pool, AntPool and GHash.IO) controlled 50% of the mining power for the past 24 hours, so Eyal is right to say:
Gavin Andresen, chief scientist of the Bitcoin Foundation, has repeatedly urged miners to use smaller pools, and researchers, including ourselves, have suggested technical fixes to reduce pool size (here, and here). But alas, community pressure has only had limited success, and technical solutions are still under development and far from production.
Eyal's post, and the detailed analysis in, are important because they show how the block withholding attack on mining pools that has been known since 2011, and has been used at least once in practice, can create a countervailing pressure that would limit the size of mining pools. Below the fold I discuss the details and the implications for my analysis.

Tuesday, January 6, 2015

Stretching the "peer reviewed" brand until it snaps

The very first post to this blog, seven-and-a-half years and 265 posts ago, was based on an NSF/JISC workshop on scholarly communication. I expressed skepticism about the value added by peer review, following Don Waters by quoting work from Diane Harley et al:
They suggest that "the quality of peer review may be declining" with "a growing tendency to rely on secondary measures", "difficult[y] for reviewers in standard fields to judge submissions from compound disciplines", "difficulty in finding reviewers who are qualified, neutral and objective in a fairly closed academic community", "increasing reliance ... placed on the prestige of publication rather than ... actual content", and that "the proliferation of journals has resulted in the possibility of getting almost anything published somewhere" thus diluting "peer-reviewed" as a brand.
My prediction was:
The big problem will be a more advanced version of the problems currently plaguing blogs, such as spam, abusive behavior, and deliberate subversion.
Since then, I've returned to the theme at intervals, pointing out that reviewers for top-ranked journals fail to perform even basic checks, that the peer-reviewed research on peer review shows that the value even top-ranked journals add is barely detectable, even before allowing for the value subtracted by their higher rate of retraction, and that any ranking system for journals is fundamentally counter-productive. As recently as 2013 Nature published a special issue on scientific publishing that refused to face these issues by failing to cite the relevant research. Ensuring relevant citation is supposed to be part of the value top-ranked journals add.

Recently, a series of incidents has made it harder for journals to ignore these problems. Below the fold, I look at some of them.

Friday, December 26, 2014

Crypto-currency as a basis for preservation

Although I have great respect for the technology underlying crypto-currencies such as Bitcoin, I've been skeptical for some time as to its viability as a product in the market both as a currency and as the basis for peer-to-peer storage proposals such as Permacoin and MaidSafe. The attraction of crypto-currencies is their decentralized nature, but if they become successful enough to be generally useful, economies of scale lead to their centralization. It was easy to get caught up in the enthusiasm as Bitcoin grew rapidly, but:
Bitcoin was the worst investment of 2014, as its value halved.
Bitcoin's hash rate had been growing exponentially since the start of 2013 but has been approximately flat for the last quarter, indicating that investment in new mining hardware has dried up.
The reason for investment drying up is likely that the revenue from mining is less than a third of what it was.
The Bitcoin market capitalization dropped from $11B to $4.4B.
Even if you don't accept my economies of scale arguments, these numbers should temper your enthusiasm for basing peer-to-peer storage on a crypto-currency.

Thursday, December 18, 2014

Economic Failures of HTTPS

Bruce Schneier points me to Assessing legal and technical solutions to secure HTTPS, a fascinating, must-read analysis of the (lack of) security on the Web from an economic rather than a technical perspective by Axel Arnbak and co-authors from Amsterdam and Delft universities. Do read the whole paper, but below the fold I provide some choice snippets.

Tuesday, December 16, 2014

Hardware I/O Virtualization

At, Timothy Prickett Morgan has an interesting post entitled A Rare Peek Into The Massive Scale Of AWS. It is based on a talk by Amazon's James Hamilton at the re:Invent conference. Morgan's post provides a hierarchical, network-centric view of the AWS infrastructure:
  • Regions, 11 of them around the world, contain Availability Zones (AZ).
  • The 28 AZs are arranged so that each Region contains at least 2 and up to 6 datacenters.
  • Morgan estimates that there are close to 90 datacenters in total, each with 2000 racks, burning 25-30MW.
  • Each rack holds 25 to 40 servers.
AZs are no more than 2ms apart measured in network latency, allowing for synchronous replication. This means the AZs in a region are only a couple of kilometres apart, which is less geographic diversity than one might want, but a disaster still has to have a pretty big radius to take out more than one AZ. The datacenters in an AZ are not more than 250us apart in latency terms, close enough that a disaster might take all the datacenters in one AZ out.

Below the fold, some details and the connection between what Amazon is doing now, and what we did in the early days of NVIDIA.