I was asked to summarize what would be needed for success apart from working technology (which we pretty much have)? My answer was four things:
- A sustainable business model
- Anti-trust enforcement
- The killer app
- A way to remove content
What Is This "Success" Of Which You Speak?
Lets arbitrarily set a target for the decentralized Web of 1% market share. That would be the equivalent of:- Decentralized social networks with, in aggregate, 22M monthly active users (Facebook has 2.23B, Snapchat has around 190M).
- Decentralized search handling in aggregate 35M searches/day (Google handles 3.5B for a 90% market share, Bing handles about 120M).
- Decentralized messaging handling in aggregate 5M messages/day (Twitter handles 500M/day).
- Decentralized video streaming in aggregate 10M hours/day (YouTube streams 1B for about 80% market share, Vimeo has around 1%).
Something around one-fifth the size of Snapchat and Bing, and about the size of Vimeo, would represent a significant market penetration, despite the fact that 99% of all Web use would still be centralized.
Sustainable Business Model
 |
| Source |
A convincing pitch to the funders needs to include a sustainable business model. What ideas for such models are being floated?
- The Web we have is powered by advertising, some $200B/year of it. The way almost all Web advertising works now involves tracking users and accumulating detailed information about them. Economies of scale mean that systems to do this will be centralized. The decentralized Web concept is that users would store their own data, and advertisers would buy it from them in some redacted or anonymized form via micro-payments. But advertisers don't want to deal with billions of individual suppliers. They want to buy access to aggregated data from a single, or at most a few suppliers. They're the ones with the money, so what they want is important. Users are the ones with the eyeballs, and they hate advertising so much that they have mounted the "biggest consumer boycott in history" by installing ad-blockers.
- For more than two decades people have been seeking the Holy Grail of micro-payments, but without success. Based on experience Clay Shirky in 2000 and Andrew Odlyzko in 2003 demolished the idea. But it is hard to kill. In 2008 Satoshi Nakamoto designed Bitcoin to handle micro-payments. It can't handle payments, much less micro-payments. In 2018 the Lightning Network was intended to be an overlay on the Bitcoin network to fix this problem. It doesn't work, and isn't decentralized.
- If you believe crypto-currency "prices", blockchain startups have been raising ludicrous sums via Initial Coin Offerings (ICOs), for example FileCoin's $257M. But note the "Initial". ICOs are a way to fund a startup by getting customers to pre-pay for services they hope eventually to obtain from the startup. This is almost the inverse of sustainable. After all the ICO proceeds have been spent building the service it has to be provided to the original customers for free.
- DuckDuckGo has a sustainable business model based on advertising that doesn't involve tracking or collecting information on users:
"This keyword-based advertising is our primary business model. When you search on DuckDuckGo, we can show you an ad based on the keywords you type in. That’s it. And it works. Our privacy policy, in a nutshell, is to not collect or share any personal information at all. Every time you search on DuckDuckGo it is as if you were there for the first time – anonymous."
At first glance this seems like a more decentralization-friendly business model, but there are two problems with it. First, DuckDuckGo is a central point from which advertisers can buy an aggregated audience. Second, search engines are special because their users tell them explicitly what they are interested in, and thus what ads to show. This isn't true of websites in general.
Anti-Trust Enforcement
 |
| Source |
It turns out that startups are Trojan horses. We think of them as revolutionaries when in fact, they’re the farm team for the establishment.As the chart from Steve Faktor shows, these days startups get bought by the incumbent giants before they can become big and very profitable, and thus pose a threat to the incumbents.
At present the entire universe of decentralized apps on Ethereum has about 8.5K daily users and revenue of about $2.5K/day. Diaspora, an open source decentralized social network has 0.004% of the users Facebook has. Things like this aren't worth noticing, let alone buying.
But if the aggregate of the new, small decentralized companies closed in on my definition of success, the incumbents would buy them up and kill them off. What's to stop them? As Cory Doctorow explained in Big Tech's problem is Big, not Tech, his barn-burner of a closing talk at the recent Decentralized Web Summit, anti-trust law is the only thing that could stop them. But decades of Chicago-school economics have gutted anti-trust enforcement, so there really isn't anything stopping the incumbents continuing to buy up and kill off the competition.
Reviving anti-trust enforcement is a difficult political problem, but there are some hopeful signs. Lina M. Kahn's Amazon's Antitrust Paradox seems to have triggered an avalanche of concern, ranging from The Economist to Chicago's Booth School.
The Killer App
Apart from a business model, another thing our new small company needs is a product., or rather a service. In What do you believe now that you didn’t five years ago? Centralized wins. Decentralized loses scalability expert Todd Hoff wrote:Decentralized systems will continue to lose to centralized systems until there’s a driver requiring decentralization to deliver a clearly superior consumer experience. Unfortunately, that may not happen for quite some time.Hoff lays out the reasons why the overheads of decentralization ensure that centralization wins. The killer app will not be "[centralized app] but decentralized", because it won't be as good as [centralized app]. Even if it were, these days who needs Second Life, let alone "Second Life, but on the blockchain"? It has to be something that users need, but that cannot be implemented by a centralized system. Hoff has a list of 6 potential drivers for this, but he's skeptical of all of them and so am I.
A Way To Remove Content
Properly implemented, the decentralized Web is content-addressed; content is named by what it is not where it is, as in the Web we have. A request for a named unit of content may be satisfied by any of possibly many copies. Among the important benefits this brings is that caching works right, something that is very hard to do in the Web we have.Content-addressing is what makes the decentralized Web's touted censorship-resistance work. A censor has to find all the copies of the content to be suppressed, and destroy all of them. Because of the way content-addressed networks work, the act of looking for the copies will probably cause more copies to be created. Much of what's driving the decentralized Web is concerns about the way the Web we have is censored not just by governments but also by the incumbent platforms.
Unfortunately, politicians love to pose as defending their constituents from bad people by passings laws censoring content on the Web, preferably by forcing the incumbent platforms to do it for them. Laws against child porn and "terrorism", and for the "right to be forgotten", "protection" of personal information, and "protection" of intellectual property all require Web publishing systems to implement some means for removing content.
Removing content from a decentralized, content-addressed system is difficult to implement:
- At the organizational level, who is the censor in a decentralized Web to whom the request to remove content is addressed, and who decides whether the request is valid? Facebook has a vast team of content moderators failing to purge deprecated content. Google has a whole organization handling DMCA takedown requests. How are their equivalents in the decentralized Web to be implemented and paid for?
- At the technical level, it is close to impossible to prevent abuse of the mechanism for tracking down and deleting all copies of some named content the censors need to do their job. Or even a mechanism for just de-indexing it.
One of the problems is that it has become almost impossible to bootstrap a major project like this. Once upon a time, you could get part of it working and convince a few customers to pay you for the value that adds. You could then use the revenue to build the other parts that were still not implemented.
ReplyDeleteToday, customers are only interested in feature-complete, fully-tested in an enterprise environment, and proven scalable software products. In order to build that, you have to get a venture capitalist to fund you something like $5-10 million up front. They are extremely reluctant to do that until you have paying customers. A classic catch-22 situation.
I am building such a system, but I have been at it for over 7 years and it is still only about half complete. Luckily, I am wealthy enough to do it without an income, but if I needed one I would have given up years ago.
DidgetMaster, you are missing a whole list of points. Decentralizing the Web is not "a major project" that could be funded by VCs:
ReplyDelete1. The whole point of a decentralized Web is that it not be dominated by a single technology or organization. So it has to be a whole ecosystem of small projects, not a few big ones.
2. In Silicon Valley the VCs mantra is "success is good, failure is OK, not doing either is a big problem". VCs funding projects in the decentralized Web really do not want to end up having funded a company with a small market share in its market. They want either success (a huge market share) or failure (no market share). They fundamentally do not want decentralization; their goals are in conflict with those of the founders.
3. As I've been pointing out in, for example, It Isn't About The Technology, decentralization is a marketing, business and governance issue, not a technology one. It doesn't matter how "feature-complete, fully-tested in an enterprise environment, and proven scalable" your technology is if it doesn't deliver capabilities that customers will pay for, at a price that will sustain a business, and (for decentralization) with a governance structure that can address real-world issues (i.e. not like Bitcoin's).
4. The major funding source for decentralized projects is ICOs, not VCs. No ICO investors expect the service to be "feature-complete, fully-tested in an enterprise environment, and proven scalable". VCs involved in ICO pre-sales are cynically preying on the uninformed.
I'm sorry you're so unhappy that after 7 years work you're still half done. Maybe you should wonder whether a system with an architecture a decade and a half old at launch might not then be obsolete. You need to post future complaints of this kind elsewhere.
David, I am sorry if my previous post came across as complaining too much. In reality, I am not unhappy at all. In fact I am more excited than ever about my project. I am a bit disappointed, however; that it is not as far along as it would have been with more funding.
ReplyDeleteMy point was that any software project that is bigger than one or two man-years; generally requires some kind of external funding. Customers, VCs, and others seem to be increasingly reluctant to provide any funding before it is largely finished.
I am building a global data network that consists of millions (or billions) of individual data nodes that are capable of exchanging data between them in a peer-to-peer fashion. Each node is basically a kind of "object store" that can manage anything from just a few data objects on simple devices to hundreds of millions of them on powerful computer systems.
I have concentrated so far on adding value at the node level. This means that if I just had a laptop not connected to the network with one of these nodes on it; that I would want my data in that node rather than in the local file system, a local database, or other NoSQL solution. My data would be more secure; easier to find and organize; faster to access; and otherwise much more useful than if it were in one of those other systems.
I am convinced that this kind of thing is necessary to successfully build a distributed data system. Just building a system that behaves exactly like your local file system (only your data is spread out across lots of nodes) is not going to drive enough adoption for widespread success.
Maybe I will be proven wrong, but I am pressing forward nevertheless. I have thought a little about some kind of ICO to provide some needed funding, but maybe I need to pursue it more earnestly. I don't think the delay is causing obsolescence (so far). I have yet to see anything close to its capabilities. It can find file data thousands of times faster than existing file systems and it does queries against relational tables about twice as fast as other databases.
DidgetMaster, fine. You think in 7 years you will have killer technology. But It Isn't About The Technology. Perhaps you might address the four issues the post you are commenting on is about.
ReplyDeleteDavid, I was following several of your blog posts and just made a general comment in one of them. Sorry if I didn't address it directly.
ReplyDeleteI disagree with your statements: "I was asked to summarize what would be needed for success apart from working technology (which we pretty much have)" and "But it isn't about the technology". I think it really IS about the technology (which we do not yet have).
We certainly have technology that will chop your file data into thousands of redundant shards and spread them across a set of distributed nodes. That will certainly distribute the data, but it is not what is needed to gain widespread user adoption.
What we need is a system that uses a distributed model as its underlying data layer, but also makes every piece of data more valuable simply because each is a data object within that system. Users must see a huge value-add with data features beside just distribution.
For example, my picture must be more valuable because it is stored and tagged as a data object within a collection of my data nodes rather than as just another file within one of my file systems. My customer list must be more valuable because it is a set of data objects that can be easily found and accessed by a variety of applications instead of hidden away in a database table that requires special connections and logins. My semi-structured data must be more valuable because it can be imported and analyzed quickly in a more flexible structure, rather than just stored within something like MongoDB.
It is technology that will make this work. My system addresses 3 of your 4 points (the anti-trust item being the exception). If it becomes a success, the only thing stopping a big corporate buyout, is the founders being willing to turn down a huge offer by putting principle over price (not easy, but possible).
P.S. I think we can get there much sooner than 7 years. We already have a few customers who see value in the stuff we have working so far.
Recently, I've come across "The Evolutionary Origins of Hierarchy" paper, where the authors investigate the hypothesis: "the existence of costs for network connections creates indirect selection for the evolution of hierarchy". That paper, for some reason, really highlighted to me that there seems to be something about the structure of the universe which seems to make decentralized system untenable universally. The many arguments in your blog posts provide ample supporting evidence for this.
ReplyDeleteHaving said that, there does seem to be a place in the universe for decentralized systems, but, the constraint seems to be specific scales in time and space. Have you explored enumerating the time scales and spatial scales where decentralized systems have selective advantage?
You have enumerated properties that a decentralized system would need to "win" at in order to have selective advantage. It seems "humans doing human things on Earth" time scale / spatial scale will not afford "win" conditions for decentralized systems. What scales would?
As regards "abuse of the mechanism for tracking down and deleting all copies of some named content":
ReplyDelete"James Rhodes, a pianist, performed a Bach composition for his Youtube channel, but it didn't stay up -- Youtube's Content ID system pulled it down and accused him of copyright infringement because Sony Music Global had claimed that they owned 47 seconds' worth of his personal performance of a song whose composer has been dead for 300 years."
From Cory Doctorow's The future is here today: you can't play Bach on Youtube because Sony says they own his compositions.
DidgetMaster:
ReplyDelete1) You are proposing a system that is distributed, but it isn't decentralized because you control it.
2) You are proposing a system that isn't an application, it is infrastructure for applications. So someone needs to develop a killer app on top of it.
3) You want funding. The idea that funders or even founders would turn down the kind of money that the FAANGs can dump on a threat is not credible.
This discussion is now closed.
More on "abuse of the mechanism for tracking down and deleting all copies of some named content" in Mike Masnick's Google Moderation Team Decides My Piece About The Impossible Nature Of Content Moderation Is 'Dangerous Or Derogatory'.
ReplyDeleteNathaniel Popper's For Big Tech, a Comeuppance We’ve Seen Before: On Wall St. is a good description of the way the FAANGs have lost the public's trust, and how the proposed cures, like the EU's copyright proposal are likely to make things worse, cementing the dominance of the FAANGs by imposing impossibly expensive requirements on potential competitors.
ReplyDeleteZheping Wang's China requires blockchain-based information service providers to register users using real names, censor postings and store user data illustrates that one of the big problems facing blockchain-based systems in the real world is governments:
ReplyDelete"Under proposed new rules, companies and entities operating in China that provide blockchain-based information services will have to ask users to register their real names and national identification card numbers, censor content deemed to pose a threat to national security and store user data to allow inspection by authorities."
The trigger was when:
"an activist in China published an open letter in April about an alleged cover-up of sexual harassment at a top university more than two decades ago on the ethereum blockchain, after the post attracted censors on social media platforms such as WeChat and Weibo."
Irina Bolychevsky's How solid is Tim’s plan to redecentralize the web? is an interesting take on the decentralized Web, making points related to those in this post.
ReplyDeleteDewayne Hendricks makes an important point about the decentralized Web in Lisa Lacy's The 10 Most Important Tech Stories of 2018:
ReplyDelete"It’s easy to go to the Dark Side, but to become a Jedi, you have to train for years to develop a set of skills. That’s what [Berners-Lee] is asking people to do—to become Jedis."
Recent research reinforces the importance of economies of scale in IT. From the conclusion to The return on information technology by Emmanuel Dhyne et al:
ReplyDelete"While all firms benefit from IT, it is the largest firms that benefit most. Furthermore, the gross marginal product of IT capital is relatively high, with €1.5 added value for €1 of extra investment in the majority of relatively low-productivity industries."
Mike Masnick's Gab, Mastodon And The Challenges Of Content Moderation On A More Distributed Social Network is a very interesting look at the repercussions of the decision of Gab, the notorious right-wing trolling site, to move to Mastodon, the social network software that is sometimes called decentralized but is more accurately called federated:
ReplyDelete"The switch over happened earlier this month and it's been fascinating to watch how it's all played out. The shortest summary might be that the federated model has shown to be somewhat resilient so far. Mastodon itself put out a statement urging various Mastodon instances not to federate with Gab and also suggesting that the various Mastdodon app developers choose to blacklist Gab's domains from their apps (meaning that Gab's plan to use this to get back into the app store might not work as well as planned).
The Verge has a long, in-depth article about how all of this is playing out, and it seems like, as a federated system is designed to do, different parts of the system are experimenting and figuring out what makes sense. Most of the other instances have decided they don't want to federate with Gab.