Tuesday, July 20, 2021

Alternatives To Proof-of-Work

The designers of peer-to-peer consensus protocols such as those underlying cryptocurrencies face three distinct problems. They need to prevent:
  • Being swamped by a multitude of Sybil peers under the control of an attacker. This requires making peer participation expensive, such as by Proof-of-Work (PoW). PoW is problematic because it has a catastrophic carbon footprint.
  • A rational majority of peers from conspiring to obtain inappropriate benefits. This is thought to be achieved by decentralization, that is a network of so many peers acting independently that a conspiracy among a majority of them is highly improbable. Decentralization is problematic because in practice all successful cryptocurrencies are effectively centralized.
  • A rational minority of peers from conspiring to obtain inappropriate benefits. This requirement is called incentive compatibility. This is problematic because it requires very careful design of the protocol.
In the rather long post below the fold I focus on some potential alternatives to PoW, inspired by Jeremiah Wagstaff's Subspace: A Solution to the Farmer’s Dilemma, the white paper for a new blockchain technology.

Careful design of the economic mechanisms of the protocol can in theory ensure incentive compatibility, or as Ittay Eyal and Emin Gun Sirer express it:
the best strategy of a rational minority pool is to be honest, and a minority of colluding miners cannot earn disproportionate benefits by deviating from the protocol
They showed in 2013 that the Bitcoin protocol was not incentive-compatible, but this is in principle amenable to a technical fix. Unfortunately, ensuring decentralization is a much harder problem.

Decentralization

Vitalik Buterin, co-founder of Ethereum, wrote in The Meaning of Decentralization:
In the case of blockchain protocols, the mathematical and economic reasoning behind the safety of the consensus often relies crucially on the uncoordinated choice model, or the assumption that the game consists of many small actors that make decisions independently.
The Internet's basic protocols, TCP/IP, DNS, SMTP, HTTP are all decentralized, and yet the actual Internet is heavily centralized around a few large companies. Centralization is an emergent behavior, driven not by technical but by economic forces. W. Brian Arthur described these forces before the Web took off in his 1994 book Increasing Returns and Path Dependence in the Economy.

Similarly, the blockchain protocols are decentralized but ever since 2014 the Bitcoin blockchain has been centralized around 3-4 large mining pools. Buterin wrote:
can we really say that the uncoordinated choice model is realistic when 90% of the Bitcoin network’s mining power is well-coordinated enough to show up together at the same conference?
This is perhaps the greatest among the multiple failures of Satoshi Nakamoto's goals for Bitcoin. The economic forces driving this centralization are the same as those that centralized other Internet protocols. I explored how they act to centralize P2P systems in 2014's Economies of Scale in Peer-to-Peer Networks. I argued that an incentive-compatible protocol wasn't adequate to prevent centralization. The simplistic version of the argument was:
  • The income to a participant in an incentive-compatible P2P network should be linear in their contribution of resources to the network.
  • The costs a participant incurs by contributing resources to the network will be less than linear in their resource contribution, because of the economies of scale.
  • Thus the proportional profit margin a participant obtains will increase with increasing resource contribution.
  • Thus the effects described in Brian Arthur's Increasing Returns and Path Dependence in the Economy will apply, and the network will be dominated by a few, perhaps just one, large participant.
And I wrote:
The advantages of P2P networks arise from a diverse network of small, roughly equal resource contributors. Thus it seems that P2P networks which have the characteristics needed to succeed (by being widely adopted) also inevitably carry the seeds of their own failure (by becoming effectively centralized). Bitcoin is an example of this.
My description of the fundamental problem was:
The network has to arrange not just that the reward grows more slowly than the contribution, but that it grows more slowly than the cost of the contribution to any participant. If there is even one participant whose rewards outpace their costs, Brian Arthur's analysis shows they will end up dominating the network. Herein lies the rub. The network does not know what an individual participant's costs, or even the average participant's costs, are and how they grow as the participant scales up their contribution.

So the network would have to err on the safe side, and make rewards grow very slowly with contribution, at least above a certain minimum size. Doing so would mean few if any participants above the minimum contribution, making growth dependent entirely on recruiting new participants. This would be hard because their gains from participation would be limited to the minimum reward. It is clear that mass participation in the Bitcoin network was fuelled by the (unsustainable) prospect of large gains for a small investment.
The result of limiting reward growth would be a blockchain with limited expenditure on mining which, as we see with the endemic 51% attacks against alt-coins, would not be secure. But without such limits, economies of scale mean that the blockchain would be dominated by a few large mining pools, so would not be decentralized and would be vulnerable to insider attacks. Note that in June 2014 the GHash.io mining pool alone had more than 51% of the Bitcoin mining power.

But the major current problem for Bitcoin, Ethereum and cryptocurrencies in general is not vulnerability to 51% attacks. Participants in these "trustless" systems trust that the mining pools are invested in their security and will not conspire to misbehave. Events have shown that this trust is misplaced as applied to smaller alt-coins. Trustlessness was one of Nakamoto's goals, another of the failures. But as regards the major cryptocurrencies this trust is plausible; everyone is making enough golden eggs to preserve the life of the goose.

Alternatives to Proof-of-Work

The major current problem for cryptocurrencies is that their catastrophic carbon footprint has attracted attention. David Gerard writes:
The bit where proof-of-work mining uses a country’s worth of electricity to run the most inefficient payment system in human history is finally coming to public attention, and is probably Bitcoin’s biggest public relations problem. Normal people think of Bitcoin as this dumb nerd money that nerds rip each other off with — but when they hear about proof-of-work, they get angry. Externalities turn out to matter.
Yang Xiao et al's A Survey of Distributed Consensus Protocols for Blockchain Networks is very useful. They:
identify five core components of a blockchain consensus protocol, namely, block proposal, block validation, information propagation, block finalization, and incentive mechanism. A wide spectrum of blockchain consensus protocols are then carefully reviewed accompanied by algorithmic abstractions and vulnerability analyses. The surveyed consensus protocols are analyzed using the five-component framework and compared with respect to different performance metrics.
Their "wide spectrum" is comprehensive as regards the variety of PoW protocols, and as regards the varieties of Proof-of-Stake (PoS) protocols that are the leading alternatives to PoW. Their coverage of other consensus protocols is less thorough, and as regards the various protocols that defend against Sybil attacks by wasting storage instead of computation it is minimal.

The main approach to replacing PoW with something equally good at preventing Sybil attacks but less good at cooking the planet has been PoS, but a recent entrant using Proof-of-Time-and-Space (I'll use PoTaS since the acronyms others use are confusing) to waste storage has attracted considerable attention. I will discuss PoS in general terms and two specific systems, Chia (PoTaS) and Subspace (a hybrid of PoTaS and PoS).

Proof-of-Stake

In PoW as implemented by Nakamoto, the probability of a winning the next block is proportional to the number of otherwise useless hashes computed — Nakamoto thought by individual CPUs but now by giant mining pools driven by warehouses full of mining ASICs. The idea of PoS is that the resource being wasted to deter Sybil attacks is the cryptocurrency itself. In order to mount a 51% attack the attacker would have to control more of the cryptocurrency that the loyal peers. In vanilla PoS the probability of winning the next block is proportional to the amount of the cryptocurrency "staked", i.e. effectively escrowed and placed at risk of being "slashed" if the majority concludes that the peer has misbehaved. It appears to have been first proposed in 2011 by Bitcointalk user QuantumMechanic.

The first cryptocurrency to use PoS, albeit as a hybrid with PoW, was Peercoin in 2012. There have been a number of pure PoS cryptocurrencies since, including Cardano from 2015 and Algorand from 2017 but none have been very successful.

Ethereum, the second most important cryptocurrency, understood the need to replace PoW in 2013 and started work in 2014. But as Vitalik Buterin then wrote:
Over the last few months we have become more and more convinced that some inclusion of proof of stake is a necessary component for long-term sustainability; however, actually implementing a proof of stake algorithm that is effective is proving to be surprisingly complex.

The fact that Ethereum includes a Turing-complete contracting system complicates things further, as it makes certain kinds of collusion much easier without requiring trust, and creates a large pool of stake in the hands of decentralized entities that have the incentive to vote with the stake to collect rewards, but which are too stupid to tell good blockchains from bad.
Buterin was right about making "certain kinds of collusion much easier without requiring trust". In On-Chain Vote Buying and the Rise of Dark DAOs Philip Daian and co-authors show that "smart contracts" provide for untraceable on-chain collusion in which the parties are mutually pseudonymous. It is obviously much harder to prevent bad behavior in a Turing-complete environment. Seven years later Ethereum is still working on the transition, which they currently don't expect to be complete for another 18 months:
Shocked to see that the timeline for Ethereum moving to ETH2 and getting off proof-of-work mining has been put back to late 2022 … about 18 months from now. This is mostly from delays in getting sharding to work properly. Vitalik Buterin says that this is because the Ethereum team isn’t working well together. [Tokenist]
Skepticism about the schedule for ETH2 is well-warranted, as Julia Magas writes in When will Ethereum 2.0 fully launch? Roadmap promises speed, but history says otherwise:
Looking at how fast the relevant updates were implemented in the previous versions of Ethereum roadmaps, it turns out that the planned and real release dates are about a year apart, at the very minimum.
Are there other reasons why PoS is so hard to implement safely? Bram Cohen's talk at Stanford included a critique of PoS:
  • Its threat model is weaker than Proof of Work.
  • Just as Proof of Work is in practice centralized around large mining pools, Proof of Stake is centralized around large currency holdings (which were probably acquired much more cheaply than large mining installations).
  • The choice of a quorum size is problematic. "Too small and it's attackable. Too large and nothing happens." And "Unfortunately, those values are likely to be on the wrong side of each other in practice."
  • Incentivizing peers to put their holdings at stake creates a class of attacks in which peers "exaggerate one's own bonding and blocking it from others."
  • Slashing introduces a class of attacks in which peers cause others to be fraudulently slashed.
  • The incentives need to be strong enough to overcome the risks of slashing, and of keeping their signing keys accessible and thus at risk of compromise.
  • "Defending against those attacks can lead to situations where the system gets wedged because a split happened and nobody wants to take one for the team"
Cohen seriously under-played PoS's centralization problem. It isn't just that the Gini coefficients of cryptocurrencies are extremely high, but that this is a self-reinforcing problem. Because the rewards for mining new blocks, and the fees for including transactions in blocks, flow to the HODL-ers in proportion to their HODL-ings, whatever Gini coefficient the systems starts out with will always increase. As I wrote, cryptocurrencies are:
a mechanism for transferring wealth from later adopters, called suckers, to early adopters, called geniuses.
PoS makes this "ratchet" mechanism much stronger than PoW, and thus renders them much more vulnerable to insider 51% attacks. I discussed one such high-profile attack by Justin Sun of Tron on the Steemit blockchain in Proof-of-Stake In Practice :
One week later, on March 2nd, Tron arranged for exchanges, including Huobi, Binance and Poloniex, to stake tokens they held on behalf of their customers in a 51% attack:
According to the list of accounts powered up on March. 2, the three exchanges collectively put in over 42 million STEEM Power (SP).

With an overwhelming amount of stake, the Steemit team was then able to unilaterally implement hard fork 22.5 to regain their stake and vote out all top 20 community witnesses – server operators responsible for block production – using account @dev365 as a proxy. In the current list of Steem witnesses, Steemit and TRON’s own witnesses took up the first 20 slots.
Although this attack didn't provide Tron with an immediate monetary reward, the long term value of retaining effective control of the blockchain was vastly greater than the cost of staking the tokens. I've been pointing out that the high Gini coefficients of cryptocurrencies means Proof-of-Stake centralizes control of the blockchain in the hands of the whales since 2017's Why Decentralize? quoted Vitalik Buterin pointing out that a realistic scenario was:
In a proof of stake blockchain, 70% of the coins at stake are held at one exchange.
Or in this case three exchanges cooperating.
Note that economic analyses of PoS, such as More (or less) economic limits of the blockchain by Joshua Gans and Neil Gandal, assume economically rational actors care about the iliquidity of staked coins and the foregone interest. But true believers in "number go up" have a long-term perspective similar to Sun's. The eventual progress of their coin "to the moon!" means that temporary, short-term costs are irrelevant to long-term HODL-ers.

Jude C. Nelson amplifies the centralization point:
PoW is open-membership, because the means of coin production are not tied to owning coins already. All you need to contribute is computing power, and you can start earning coins at a profit.

PoS is closed-membership with a veneer of open-membership, because the means of coin production are tied to owning a coin already. What this means in practice is that no rational coin-owner is going to sell you coins at a fast enough rate that you'll be able to increase your means of coin production. Put another way, the price you'd pay for the increased means of coin production will meet or exceed the total expected revenue created by staking those coins over their lifetime. So unless you know something the seller doesn't, you won't be able to profit by buying your way into staking.

Overall, this makes PoS less resilient and less egalitarian than PoW. While both require an up-front capital expenditure, the expenditure for PoS coin-production will meet or exceed the total expected revenue of those coins at the point of sale. So, the system is only as resilient as the nodes run by the people who bought in initially, and the only way to join later is to buy coins from people who want to exit (which would only be viable if these folks believed the coins are worth less than what you're buying them for, which doesn't bode well for you as the buyer).
Nelson continues:
PoW requires less proactive trust and coordination between community members than PoS -- and thus is better able to recover from both liveness and safety failures -- precisely because it both (1) provides a computational method for ranking fork quality, and (2) allows anyone to participate in producing a fork at any time. If the canonical chain is 51%-attacked, and the attack eventually subsides, then the canonical chain can eventually be re-established in-band by honest miners simply continuing to work on the non-attacker chain. In PoS, block-producers have no such protocol -- such a protocol cannot exist because to the rest of the network, it looks like the honest nodes have been slashed for being dishonest. Any recovery procedure necessarily includes block-producers having to go around and convince people out-of-band that they were totally not dishonest, and were slashed due to a "hack" (and, since there's lots of money on the line, who knows if they're being honest about this?).
PoS conforms to Mark 4:25:
For he that hath, to him shall be given: and he that hath not, from him shall be taken even that which he hath.
In Section VI(E) Yang Xiao et al identify the following types of vulnerability in PoS systems:
  1. Costless simulation:
    literally means any player can simulate any segment of blockchain history at the cost of no real work but speculation, as PoS does not incur intensive computation while the blockchain records all staking history. This may give attackers shortcuts to fabricate an alternative blockchain.
    It is the basis for attacks 2 through 5.
  2. Nothing at stake
    Unlike a PoW miner, a PoS minter needs little extra effort to validate transactions and generate blocks on multiple competing chains simultaneously. This “multi-bet” strategy makes economical sense to PoS nodes because by doing so they can avoid the opportunity cost of sticking to any single chain. Consequently if a significantly fraction of nodes perform the “multi-bet” strategy, an attacker holding far less than 50% of tokens can mount a successful double spending attack.
    The defense against this attack is usually "slashing", forfeiting the stake of miners detected on multiple competing chains. But slashing, as Cohen and Nelson point out, is in itself a consensus problem.
  3. Posterior corruption
    The key enabler of posterior corruption is the public availability of staking history on the blockchain, which includes stakeholder addresses and staking amounts. An attacker can attempt to corrupt the stakeholders who once possessed substantial stakes but little at present by promising them rewards after growing an alternative chain with altered transaction history (we call it a “malicious chain”). When there are enough stakeholders corrupted, the colluding group (attacker and corrupted once-rich stakeholders) could own a significant portion of tokens (possibly more than 50%) at some point in history, from which they are able to grow an malicious chain that will eventually surpass the current main chain.
    The defense is key-evolving cryptography, which ensures that the past signatures cannot be forged by the future private keys.
  4. Long-range attack as introduced by Buterin:
    foresees that a small group of colluding attackers can regrow a longer valid chain that starts not long after the genesis block. Because there were likely only a few stakeholders and a lack of competition at the nascent stage of the blockchain, the attackers can grow the malicious chain very fast and redo all the PoS blocks (i.e. by costless simulation) while claiming all the historical block rewards.
    Evangelos Deirmentzoglou et al's A Survey on Long-Range Attacks for Proof of Stake Protocols provides a useful review of these attacks. Even if there are no block rewards, only fees, a variant long-range attack is possible as described in Stake-Bleeding Attacks on Proof-of-Stake Blockchains by Peter Gazi et al, and by Shijie Zhang and Jong-Hyouk Lee in Eclipse-based Stake-Bleeding Attacks in PoS Blockchain Systems.
  5. Stake-grinding attack
    unlike PoW in which pseudo-randomness is guaranteed by the brute-force use of a cryptographic hash function, PoS’s pseudo-randomness is influenced by extra blockchain information—the staking history. Malicious PoS minters may take advantage of costless simulation and other staking-related mechanisms to bias the randomness of PoS in their own favor, thus achieving higher winning probabilities compared to their stake amounts
  6. Centralization risk as discussed above:
    In PoS the minters can lawfully reinvest their profits into staking perpetually, which allows the one with a large sum of unused tokens become wealthier and eventually reach a monopoly status. When a player owns more than 50% of tokens in circulation, the consensus process will be dominated by this player and the system integrity will not be guaranteed.
    There are a number of papers on this problem, including Staking Pool Centralization in Proof-of-Stake Blockchain Network by Ping He et al, Compounding of wealth in proof-of-stake cryptocurrencies by Giulia Fanti et al, and Stake shift in major cryptocurrencies: An empirical study by Rainer Stütz et al. But to my mind none of them suggest a realistic mitigation.
These are not the only problems from which PoS suffers. Two more are:
  • Checkpointing. Long-range and related attacks are capable of rewriting almost the entire chain. To mitigate this, PoS systems can arrange for consensus on checkpoints, blocks which are subsequently regarded as canonical forcing any rewriting to start no earlier than the following block. Winkle – Decentralised Checkpointing for Proof-of-Stake is:
    a decentralised checkpointing mechanism operated by coin holders, whose keys are harder to compromise than validators’ as they are more numerous. By analogy, in Bitcoin, taking control of one-third of the total supply of money would require at least 889 keys, whereas only 4 mining pools control more than half of the hash power
    It is important that consensus on checkpoints is achieved through a different mechanism than consensus on blocks. To over-simplify, Winkle piggy-backs votes for checkpoints on transactions; a transaction votes for a block with the number of coins remaining in the sending account, and with the number sent to the receiving account. A checkpoint is final once a set proportion of the coins have voted for it. For the details, see Winkle: Foiling Long-Range Attacks in Proof-of-Stake Systems by Sarah Azouvi et al.
  • Lending. In Competitive equilibria between staking and on-chain lending, Tarun Chitra demonstrates that it is:
    possible for on-chain lending smart contracts to cannibalize network security in PoS systems. When the yield provided by these contracts is more attractive than the inflation rate provided from staking, stakers will tend to remove their staked tokens and lend them out, thus reducing network security. ... Our results illustrate that rational, non-adversarial actors can dramatically reduce PoS network security if block rewards are not calibrated appropriately above the expected yields of on-chain lending.
    I believe this is part of a fundamental problem for PoS. The token used to prevent a single attacker appearing as a multitude of independent peers can be lent, and thus the attacker can borrow a temporary majority of the stake cheaply, for only a short-term interest payment. Preventing this increases implementation complexity significantly.
In summary, despite PoS' potential for greatly reducing PoW's environmental impact and cost of defending against Sybil attacks, it has a major disadvantage. It is significantly more complex and thus its attack surface is much larger, especially when combined with a Turing-complete execution environment such as Ethereum's. It therefore needs more defense mechanisms, which increase complexity further. Buterin and the Ethereum developers realize the complexity of the implementation task they face, which is why their responsible approach is taking so long. Currently Ethereum is the only realistic candidate to displace Bitcoin, and thus reduce cryptocurrencies' carbon footprint, so the difficulty of an industrial-strength implementation of PoS for Ethereum 2.0 is a major problem.

Proof-of-Space-and-Time

Back in 2018 I wrote about Bram Cohen's PoTaS system, Chia, in Proofs of Space and Chia Network. Instead of wasting computation to prevent Sybil attacks, Chia wastes storage. Chia's "space farmers" create and store "plots" consisting of large amounts of otherwise useless data. The technical details are described in Chia Consensus. They are comprehensive and impressively well thought out.

Because, like Bitcoin, Chia is wasting a real resource to defend against Sybil attacks it lacks many of PoS' vulnerabilities. Nevertheless, the Chia protocol is significantly more complex than Bitcoin and thus likely to possess additional vulnerabilities. For example, whereas in Bitcoin there is only one role for participants, mining, the Chia protocol involves three roles:
  • Farmer, "Farmers are nodes which participate in the consensus algorithm by storing plots and checking them for proofs of space."
  • Timelord, "Timelords are nodes which participate in the consensus algorithm by creating proofs of time".
  • Full node, which involves "broadcasting proofs of space and time, creating blocks, maintaining a mempool of pending transactions, storing the historical blockchain, and uploading blocks to other full nodes as well as wallets (light clients)."
Figure 11
Another added complexity is that the Chia protocol maintains three chains (Challenge, Reward and Foliage), plus an evanescent chain during each "slot" (think Bitcoin's block time), as shown in the document's Figure 11. The document therefore includes a range of attacks and their mitigations which are of considerable technical interest.

Cohen's praiseworthy objective for Chia was to avoid the massive power waste of PoW because:
"You have this thing where mass storage medium you can set a bit and leave it there until the end of time and its not costing you any more power. DRAM is costing you power when its just sitting there doing nothing".
Alas, Cohen was exaggerating:
A state-of-the-art disk drive, such as Seagate's 12TB BarraCuda Pro, consumes about 1W spun-down in standby mode, about 5W spun-up idle and about 9W doing random 4K reads.
Which is what it would be doing much of the time while "space farming". Clearly, PoTaS uses energy, just much less than PoW. Reporting on Cohen's 2018 talk at Stanford I summarized:
Cohen's vision is of a PoSp/VDF network comprising large numbers of desktop PCs, continuously connected and powered up, each with one, or at most a few, half-empty hard drives. The drives would have been purchased at retail a few years ago.
My main criticism in those posts was Cohen's naiveté about storage technology, the storage market and economies of scale:
There would appear to be three possible kinds of participants in a pool:
  • Individuals using the spare space in their desktop PC's disk. The storage for the Proof of Space is effectively "free", but unless these miners joined pools, they would be unlikely to get a reward in the life of the disk.
  • Individuals buying systems with CPU, RAM and disk solely for mining. The disruption to the user's experience is gone, but now the whole cost of mining has to be covered by the rewards. To smooth out their income, these miners would join pools.
  • Investors in data-center scale mining pools. Economies of scale would mean that these participants would see better profits for less hassle than the individuals buying systems, so these investor pools would come to dominate the network, replicating the Bitcoin pool centralization.
Thus if Chia's network were to become successful, mining would be dominated by a few large pools. Each pool would run a VDF server to which the pool's participants would submit their Proofs of Space, so that the pool manager could verify their contribution to the pool.

The emergence of pools, and dominance of a small number of pools, has nothing to do with the particular consensus mechanism in use. Thus I am skeptical that alternatives to Proof of Work will significantly reduce centralization of mining in blockchains generally, and in Chia Network's blockchain specifically.
As I was writing the first of these posts, TechCrunch reported:
Chia has just raised a $3.395 million seed round led by AngelList’s Naval Ravikant and joined by Andreessen Horowitz, Greylock and more. The money will help the startup build out its Chia coin and blockchain powered by proofs of space and time instead of Bitcoin’s energy-sucking proofs of work, which it plans to launch in Q1 2019.
Even in 2020 the naiveté persisted, as Chia pitched the idea that space farming on a Raspberry Pi was a way to make money. It still persists, as Chia's president reportedly claims that "recyclable hard drives are entering the marketplace". But when Chia Coin actually started trading in early May 2021 the reality was nothing like Cohen's 2018 vision:
  • As everyone predicted, the immediate effect was to create a massive shortage of the SSDs needed to create plots, and the hard drives needed to store them. Even Gene Hoffman, Chia's CEO, admitted that Bitcoin rival Chia 'destroyed' hard disc supply chains, says its boss:
    Chia, a cryptocurrency intended to be a “green” alternative to bitcoin has instead caused a global shortage of hard discs. Gene Hoffman, the president of Chia Network, the company behind the currency, admits that “we’ve kind of destroyed the short-term supply chain”, but he denies it will become an environmental drain.
    The result of the spike in storage prices was a rise in the vendors stock:
    The share price of hard disc maker Western Digital has increased from $52 at the start of the year to $73, while competitor Seagate is up from $60 to $94 over the same period.
    To give you some idea of how rapidly Chia has consumed storage in the two months since launch, it is around 20% of the rate at which the entire industry produced hard disk in 2018.

  • Chia Pools
    Mining pools arose. As I write the network is storing 30.06EB of otherwise useless data, of which one pool, ihpool.com is managing 10.78EB, or 39.3%. Unlike Bitcoin, the next two pools are much smaller, but large enough so that the top four pools have 42% of the space. The network is slightly more decentralized than Bitcoin has been since 2014, and for reasons discussed below is less vulnerable to an insider 51% attack.

  • Chia "price"
    The "price" of Chia Coin collapsed, from $1934.51 at the start of trading to $165.41 Sunday before soaring to $185.78 as I write. Each circulating XCH corresponds to about 30TB. The investment in "space farming" hardware vastly outweighs, by nearly six times, the market cap of the cryptocurrency it is supporting.

  • The "space farmers" are earning $1.69M/day, or about $20/TB/year. A 10TB internal drive is currently about $300 on Amazon, so it will be about a 18 months before it earns a profit. The drive is only warranted for 3 years. But note that the warranty is limited:
    Supports up to 180 TB/yr workload rate Workload Rate is defined as the amount of user data transferred to or from the hard drive.
    Using the drive for "space farming" would likely void the warranty and, just as PoW does to GPUs, burn out the drive long before its warranted life. If you have two years, the $300 investment theoretically earns a 25% return before power and other costs.

  • But the hard drive isn't the only cost of space farming. In order to become a "space farmer" in the first place you need to create plots containing many gigabytes of otherwise useless cryptographically-generated data. You need lots of them; the probability of winning your share of the $2.74M/day is how big a fraction of the nearly 30EB you can generate and store. The 30EB is growing rapidly, so the quicker you can generate the plots, the better your chance in the near term. To do so in finite time you need in addition to the hard drive a large SSD at extra cost. Using it for plotting will void its warranty and burn it out in as little as six weeks. And you need a powerful server running flat-out to do the cryptography, which both rather casts doubt on how much less power than PoW Chia really uses, and increases the payback time significantly.

  • In my first Chia post I predicted that "space farming" would be dominated by huge data centers such as Amazon's. Sure enough, Wolfie Zhao reported on May 7th that:
    Technology giant Amazon has rolled out a solution dedicated to Chia crypto mining on its AWS cloud computing platform.

    According to a campaign page on the Amazon AWS Chinese site, the platform touts that users can deploy a cloud-based storage system in as quickly as five minutes in order to mine XCH, the native cryptocurrency on the Chia network.
    Two weeks later David Gerard reported that:
    The page disappeared in short order — but an archive exists.
    Because Chia mining trashes the drives, something else I pointed out in my first Chia post, storage services are banning users who think that renting something is a license to destroy it. In any case, 10TB of Amazon's S3 Reduced Redundancy Storage costs $0.788/day, so it would be hard to make ends meet. Cheaper storage services, such as Wasabi at $0.20/day are at considerable risk from Chia.

  • Although this isn't an immediate effect, as David Gerard writes, because creating Chia plots wears out SSDs, and Chia farming wears out hard disks:
    Chia produces vast quantities of e-waste—rare metals, assembled into expensive computing components, turned into toxic near-unrecyclable landfill within weeks.
Miners are incentivized to join pools because they prefer a relatively predictable, frequent flow of small rewards to very infrequent large rewards. The way pools work in Bitcoin and related protocols is that the pool decides what transactions are in the block it hopes to mine, and gets all the pool participants to work on that block. Thus a pool, or a conspiracy among pools, that had 51% of the mining power would have effective control over the transactions that were finalized. Because they make the decision as to which transactions happen, Nicholas Weaver argues that mining pools are money transmitters and thus subject to the AML/KYC rules. But in Chia pools work differently:
First and foremost, even when a winning farmer is using a pool, they themselves are the ones who make the transaction block - not the pool. The decentralization benefits of this policy are obvious.
The potential future downside is that while Bitcoin miners in a pool can argue that AML/KYC is the responsibility of the pool, Chia farmers would be responsible for enforcing the AML/KYC rules and subject to bank-sized penalties for failing to do so.

In Bitcoin the winning pool receives and distributes both the block reward and the (currently much smaller) transaction fees. Over time the Bitcoin block reward is due to go to zero and the system is intended to survive on fees alone. Alas, research has shown that a fee-only Bitcoin system is insecure.

Chia does things differently in two ways. First:
all the transaction fees generated by a block go to the farmer who found it and not to the pool.

Trying to split the transaction fees with the pool could result in transaction fees being paid ‘under the table’ either by making them go directly to the farmer or making an anyone can spend output which the farmer would then pay to themselves. Circumventing the pool would take up space on the blockchain. It could also encourage the emergence of alternative pooling protocols where the pool makes the transaction block which is a form of centralization we wish to avoid.
The basic argument is that in Bitcoin the 51% conspiracy is N pools where in Chia it is M farmers (M ≫ N). Chia are confident that this is safe:
This ensures that even if a pool has 51% netspace, they would also need to control ALL of the farmer nodes (with the 51% netspace) to do any malicious activity. This will be very difficult unless ALL the farmers (with the 51% netspace) downloaded the same malicious Chia client programmed by a Bram like level genius.
I'm a bit less confident because, like Ethereum, Chia has a Turing-complete programming environment. In On-Chain Vote Buying and the Rise of Dark DAOs Philip Daian and co-authors showed that "smart contracts" provide for untraceable on-chain collusion in which the parties are mutually pseudonymous. Although their conspriacies were much smaller, similar techniques might be the basis for larger attacks on blockchains with "smart contracts".

Second:
This method has the downside of reducing the smoothing benefits of pools if transaction fees come to dominate fixed block rewards. That’s never been a major issue in Bitcoin and our block reward schedule is set to only halve three times and continue at a fixed amount forever after. There will alway be block rewards to pay to the pool while transaction fees go to the individual farmers.
So unlike the Austrian economics of Bitcoin, Chia plans to reward farming by inflating the currency indefinitely, never depending wholly on fees. In Bitcoin the pool takes the whole block reward, but the way block rewards work is different too:
fixed block rewards are set to go 7/8 to the pool and 1/8 to the farmer. This seems to be a sweet spot where it doesn’t reduce smoothing all that much but also wipes out potential selfish mining attacks where someone joins a competing pool and takes their partials but doesn’t upload actual blocks when they find them. Those sort of attacks can become profitable when the fraction of the split is smaller than the size of the pool relative to the whole system.
Last I checked ihpool.com had almost 40% of the total system.

Rational economics are not in play here. "Space farming" makes sense only at scale or for the most dedicated believers in "number go up". Others are less than happy:
So I tested this Chia thing overnight. Gave it 200GB plot and two CPU threads. After 10 hours it consumed 400GB temp space, didn’t sync yet, CPU usage is always 80%+. Estimated reward time is 5 months. This isn’t green, already being centralised on large waste producing servers.
The problem for the "number go up" believers is that the "size go up" too, by about half-an-exabyte a day. As the network grows, the chance that your investment in hardware will earn a reward goes down because it represents a smaller proportion of the total. Unless "number go up" much faster than "size go up", your investment is depreciating rapidly not just because you are burning it out but because its cost-effectiveness is decaying. And as we see, "size go up" rapidly but "number go down" rapidly. And economies of scale mean that return on investment in hardware will go up significantly with the proportion of the total the farmer has. So the little guy gets the short end of the stick even if they are in a pool.

Chia's technology is extremely clever, but the economics of the system that results in the real world don't pass the laugh test. Chia is using nearly a billion dollars of equipment being paid for by inflating the currency at a rate of currently 2/3 billion dollars a year to process transactions at a rate around five billion dollars a year, a task that could probably be done using a conventional database and a Raspberry Pi. The only reason for this profligacy is to be able to claim that it is "decentralized". It is more decentralized than PoW or PoS systems, but over time economies of scale and free entry will drive the reward for farming in fiat terms down and mean that small-scale farmers will be squeezed out.

The Chia "price" chart suggests that it might have been a "list-and-dump" scheme, in which A16Z and the other VCs incentivized the miners to mine and the exchanges to list the new cryptocurrency so that the VCs could dump their HODL-ings on the muppets seduced by the hype and escape with a profit. Note that A16Z just raised a $2.2B fund dedicated to pouring money into similar schemes. This is enough to fund 650 Chia-sized ventures! (David Gerard aptly calls Andreesen Horowitz "the SoftBank of crypto") They wouldn't do that unless they were making big bucks from at least some of the ones they funded earlier. Chia's sensitivity about their PR led them to hurl bogus legal threats at the leading Chia community blog. Neither is a good look.

Subspace

As we see, the Chia network has one huge pool and a number of relatively miniscule pools. In Subspace" A Solution to the Farmer's Dilemma, Wagstaff describes the "farmer's dilemma" thus:
Observe that in any PoC blockchain a farmer is, by-definition, incentivized to allocate as much of its scarce storage resources as possible towards consensus. Contrast this with the desire for all full nodes to reserve storage for maintaining both the current state and history of the blockchain. These competing requirements pose a challenge to farmers: do they adhere to the desired behavior, retaining the state and history, or do they seek to maximize their own rewards, instead dedicating all available space towards consensus? When faced with this farmer’s dilemma rational farmers will always choose the latter, effectively becoming light clients, while degrading both the security and decentralization of the network. This implies that any PoC blockchain would eventually consolidate into a single large farming pool, with even greater speed than has been previously observed with PoW and PoS chains.
Subspace proposes to resolve this using a hybrid of PoS and PoTaS:
We instead clearly distinguish between a permissionless farming mechanism for block production and permissioned staking mechanism for block finalization.
Wagstaff describes it thus:
  1. To prevent farmers from discarding the history, we construct a novel PoC consensus protocol based on proofs-of-storage of the history of the blockchain itself, in which each farmer stores as many provably-unique replicas of the chain history as their disk space allows.
  2. To ensure the history remains available, farmers form a decentralized storage network, which allows the history to remain fully-recoverable, load-balanced, and efficiently-retrievable.
  3. To relieve farmers of the burden of maintaining the state and preforming [sic] redundant computation, we apply the classic technique in distributed systems of decoupling consensus and computation. Farmers are then solely responsible for the ordering of transactions, while a separate class of executor nodes maintain the state and compute the transitions for each new block.
  4. To ensure executors remain accountable for their actions, we employ a system of staked deposits, verifiable computation, and non-interactive fraud proofs.
Separating consensus (PoTaS) and computation (PoS) has interesting effects:
  • Like Chia, the only function of pools is to smooth out farmer's rewards. They do not compose the blocks. Pools will compete on their fees. Economics of scale mean that the larger the pool, the lower the fees it can charge. So, just like Chia, Subspace will end up with one, or only a few, large pools.
  • Like Chia, if they can find a proof, farmers assemble transactions into a block which they can submit to executors for finalization. Subspace shares with Chia the property that a 51% attack requires M farmers not N pools (M ≫ N), assuming of course no supply chain attack or abuse of "smart contracts".
  • Subspace uses a LOCKSS-like technique of electing a random subset of executors for each finalization. Because any participant can unambiguously detect fraudulent execution, and thus that the finalization of a block is fraudulent, the opportunity for bad behavior by executors is highly constrained. A conspiracy of executors has to hope that no honest executor is elected.
Like Chia, the technology is extremely clever but there are interesting economic aspects. As regards farmers, Wagstaff writes:
To ensure the history does not grow beyond total network storage capacity, we modify the transaction fee mechanism such that it dynamically adjusts in response to the replication factor. Recall that in Bitcoin, the base fee rate is a function of the size of the transaction in bytes, not the amount of BTC being transferred. We extend this equation by including a multiplier, derived from the replication factor. This establishes a mandatory minimum fee for each transaction, which reflects its perpetual storage cost. The multiplier is recalculated each epoch, from the estimated network storage and the current size of the history. The higher the replication factor, the cheaper the cost of storage per byte. As the replication factor approaches one, the cost of storage asymptotically approaches infinity. As the replication factor decreases, transaction fees will rise, making farming more profitable, and in-turn attracting more capacity to the network. This allows the cost of storage to reach an equilibrium price as a function of the supply of, and demand for, space.
There are some issues here:
  • The assumption that the market for fees can determine the "perpetual storage cost" is problematic. As I first showed back in 2011, the endowment needed for "perpetual storage" depends very strongly on two factors that are inherently unpredictable, the future rate of decrease of media cost in $/byte (Kryder rate), and the future interest rate. The invisible hand of the market for transaction fees cannot know these, it only knows the current cost of storage. Nor can Subspace management know them, to set the "mandatory minimum fee". Thus it is likely that fees will significantly under-estimate the "perpetual storage cost", leading to problems down the road.
  • The assumption that those wishing to transact will be prepared to pay at least the "mandatory minimum fee" is suspect. Cryptocurrency fees are notoriously volatile because they are based on a blind auction; when no-one wants to transact a "mandatory minimum fee" would be a deterrent, when everyone wants to fees are unaffordable. Research has shown that if fees dominate block rewards systems become unstable.
Wagstaff's paper doesn't seem to describe how block rewards work; I assume that they go to the individual farmer or are shared via a pool for smoother cash flow. I couldn't see from the paper whether, like Chia, Subspace intends to avoid depending upon fees.

As regards executors:
For each new block, a small constant number of executors are chosen through a stake-weighted election. Anyone may participate in execution by syncing the state and placing a small deposit.
But the chance that they will be elected and gain the reward for finalizing a block and generating an Execution Receipt (ER) depends upon how much they stake. The mechanism for rewarding executors is:
Farmers split transaction fee rewards evenly with all executors, based on the expected number of ERs for each block.7 For example, if 32 executors are elected, the farmer will take half of the all transaction fees, while each executor will take 1/64. A farmer is incentivized to include all ERs which finalize execution for its parent block because doing so will allow it to claim more of its share of the rewards for its own block. For example, if the farmer only includes 16 out of 32 expected ERs, it will instead receive 1/4 (not 1/2) of total rewards, while each of the 16 executors will still receive 1/64. Any remaining shares will then be escrowed within a treasury account under the control of the community of token holders, with the aim of incentivizing continued protocol development.
Although the role of executor demands significant resources, both in hardware and in staked coins, these rewards seem inadequate. Every executor has to execute the state transitions in every block. But for each block only a small fraction of the executors receive only, in the example above, 1/64 of the fees. Note also footnote 7:
7 We use this rate for explanatory purposes, while noting that in order to minimize the plutocratic nature of PoS, executor shares should be smaller in practice.
So Wagstaff expects that an executor will receive only a small fraction of a small fraction of 1/64 of the transaction fees. Even supposing the stake distribution among executors was small and even, unlikely in practice, for the random election mechanism to be effective there need to be many times 32 executors. For example, if there are 256 executors, and executors share 1/8 of the fees, each can expect around 0.005% of the fees. Bitcoin currently runs with fees less than 10% of the block rewards. If Subspace had the same split in my example executors as a class would expect around 1.2% of the block rewards, with farmers as a class receiving 100% of the block rewards plus 87.5% of the fees.

There is another problem — the notorious volatility of transaction fees set against the constant cost of running an executor. Much of the time there would be relatively low demand for transactions, so a block would contain relatively few transactions that each offered the mandatory minimum fee. Unless the fees, and especially the mandatory minimum fee, are large relative to the block reward it isn't clear why executors would participate. But fees that large would risk the instability of fee-only blockchains.

There are two other roles in Subspace, verifiers and full nodes. As regards incentivizing verifiers:
we rely on the fact that all executors may act as verifiers at negligible additional cost, as they are already required to maintain the valid state transitions in order to propose new ERs. If we further require them to reveal fraud in order to protect their own stake and claim their share of the rewards, in the event that they themselves are elected, then we can provide a more natural solution to the verifier’s dilemma.
As regards incentivizing full nodes, Wagstaff isn't clear.
In addition to executors, any full node may also monitor the network and generate fraud proofs, by virtue of the fact that no deposit is required to act as verifier.
As I read the paper, full nodes have similar hardware requirements as executors but no income stream to support them unless they are executors too.

Overall, Subspace is interesting. But the advantage from a farmer's point of view of Subspace over Chia is that their whole storage resource is devoted to farming. Everything else is not really significant, and all this would be dominated by a fairly small difference in "price". Add to that the fact that Chia has already occupied the market niche for new PoTaS systems, and has high visibility via Bram Cohen and A16Z, and the prospects for Subspace don't look good. If Subspace succeeds, economies of scale will have two effects:
  • Large pools will dominate small pools because they can charge smaller fees.
  • Large farmers will dominate small farmers because their rewards are linear in the resource they commit, but their costs are sub-linear, so their profit is super-linear. This will likely result in the most profitable, hassle-free way for smaller consumers to participate being investing in a pool rather than actually farming.

Conclusion

The overall theme is that permissionless blockchains have to make participating in consensus expensive in some way to defend against Sybils. Thus if you are expending an expensive resource economies of scale are an unavoidable part of Sybil defense. If you want to be "decentralized" to avoid 51% attacks from insiders you have to have some really powerful mechanism pushing back against economies of scale. I see three possibilities, either the blockchain protocol designers:
  1. Don't understand why successful cryptocurrencies are centralized, so don't understand the need to push back on economies of scale.
  2. Do understand the need to push back on economies of scale but can't figure out how to do it. It is true that figuring this out is incredibly difficult, but their response should be to say "if the blockchain is going to end up centralized, why bother wasting resources trying to be permissionless?" not to implement something they claim is decentralized when they know it won't be.
  3. Don't care about decentralization, they just want to get rich quick, and are betting it will centralize around them.
In most cases, my money is on #3. At least both Chia and Subspace have made efforts to defuse the worst aspects of centralization.

6 comments:

David. said...

Chris Dupres, who I should have acknowledged provided me with a key hint about Chia, posted Very detailed post on post-Proof of Work cryptocurrencies, including Chia linking here. Thanks, Chris!

David. said...

Brett Scott's I, Token is an excellent if lengthy explanation of why Bitcoin and other currencies lack "moneyness".

Blissex2 said...

«Being swamped by a multitude of Sybil peers [...] A rational majority of peers from conspiring to obtain inappropriate benefits. [...] A rational minority of peers from conspiring to obtain inappropriate benefits.»

This analysis ids very interesting and seems well done to me, but it is orthogonal to what seems to be the economic appeal of BitCoin, which is not that it is fair, decentralized, trustworthy, but the perception that:

* It is pseudonymous and does not have "know my customer" rules.
* It is worldwide so it is not subject to capital export/import restrictions.
* There is a limited number of BitCoins, so it is "guaranteed" that as demand grows the worth of each BitCoin is going to grow a lot.

BitCoin in other words is perceived as a limited-edition, pseudonymous collectible; it is considered as the convenient alternative to a pouch of diamonds for "informal" payments.

Considering that international "informal" payments often involve a 10-20% "cleanup" fee, users of BitCoin for "informal" payments don't worry too much about issues like decentralization or perfect fairness and perfect trustworthiness, as long as the BitCoin operators steal less than 10-20% it is just a cost of doing business.

David. said...

This post wasn't about Bitcoin, or the reasons people use it. It was about the design of consensus protocols for permissionless peer-to-peer systems.

And, to your point, the use of BTC for "payments" is negligible compared to its use for speculation. The use-case for BTC HODL-ers is "to the moon", and the use-case for BTC traders is its volatility and ability to be pumped-and-dumped.

Blissex2 said...

«This post wasn't about Bitcoin, or the reasons people use it. It was about the design of consensus protocols for permissionless peer-to-peer systems.»

Indeed, as I wrote your post is “orthogonal to what seems to be the economic appeal of BitCoin”. But I think that they are relevant: the question is whether “consensus protocols for permissionless peer-to-peer systems” matter if the economic incentives are not aligned with them.

Your post also mentioned repeatedly “the multiple failures of Satoshi Nakamoto's goals for Bitcoin”, which goals were economic, and accordingly you also mention as relevant “The economic forces” for those consensus protocols. I am sorry that I was not clear as to why I was writing about economic forces orthogonally to consensus protocol, given that your post also seemed to consider them relevant.

«the use of BTC for "payments" is negligible compared to its use for speculation. The use-case for BTC HODL-ers is "to the moon", and the use-case for BTC traders is its volatility and ability to be pumped-and-dumped»

Indeed currently, but again you referred several times to “Satoshi Nakamoto's goals for Bitcoin”, and those were about a payment system.

Also the current minority of users of BitCoin and other "coins" who use it as a "don't know your customer" alternative to WesternUnion or "hawala" remittances are not irrelevant, because that use of "coins" could be long term.

When doing "don't know your customer" transfers, they could be denominated in etruscan guineas or mayan jiaozis while in transit, and "coins" are just then a unit of account, and it is for that main purpose that a “consensus protocol” was designed by Satoshi Nakamoto (even if it turned out not to be that suitable).

Maybe you think that these considerations are very secondary, but I wrote about them because I think in time they will matter.

David. said...

David Gerard wrote:

"The Chia scam seems to have played out, and Chia is crashing — and Chia farmers are dumping the trashed disks on second-hand markets"

I think this is a mis-interpretation. When I wrote the above seven weeks ago the network had 30EB and the "price" was around $185. As I write this the network has 35EB and the "price" is around $260. It has been around $260 since the start of August. So not exactly "crashing".

I think we are seeing four effects:

1) Muppets who bought into the pump and thought that XCH = $1600 was the start of "to the moon". After a couple of months this hope has died.

2) Big players with economies of scale driving the smaller players to dump their hardware.

3) The SSDs are needed only to bring the HDDs on-line, and are then trashed. So network *growth* causes trashed SSDs to be dumped.

4) Normal use of HDDs by players big and small trashes them. So even if the network isn't growing there will be a flow of trashed drives.

For 3 and 4 smart players will dump their drives before they are completely dead. They will compare the value of the XCH left in their life and the price they can get, inflated by lying, they can get for dumping them.