Tuesday, November 24, 2020
The Web has dominated digital content since 1995, and in the Web world formats go obsolete very slowly, if at all, because they are in effect network protocols. The example of IPv6 shows how hard it is to evolve network protocols. But now we are facing the obsolescence of a Web format that was very widey used as the long effort to kill off Adobe's Flash comes to fruition. Fortunately, Jason Scott's Flash Animations Live Forever at the Internet Archive shows that we were right all along. Below the fold, I go into the details.
Thursday, November 19, 2020
Thursday, November 12, 2020
El Reg has heard on good authority from sources in British political circles that Cambridge Analytica's advertised powers of online suggestion were rather overblown and in fact mostly useless. In the end, it was skewered by its own hype, accused of tangibly influencing the Brexit and presidential votes on behalf of political parties and campaigners using its Facebook data. Yet, no evidence, according to the ICO, could be found supporting those specific claims.Below the fold I look at this, a recent book on the topic, and other evidence that has emerged since I wrote Contextual vs. Behavioral Advertising.
Tuesday, November 3, 2020
"It's the order flow, isn't it?" Milgrim had had no intent to ask this at all. Hadn't been thinking off it. Yet it had emerged. His therapist had told him that ideas, in human relations, had lives of their own. Were in a sense autonomous.Entirely adequate to make Hubertus Bigend much, much richer, because knowing the order flow allows him to front-run the transactions.
"That's what Chombo was doing. Finding the order flow."
"He found it a week before they kidnapped him, but his work, to that point, would have been useless, Without him, I mean."
"And the market, the whole thing, it's no longer real? Because you know the future?"
"It's a very tiny slice of the future. The merest paring. Minutes."
Bigend had glanced around the empty lounge. "Seventeen, presently."
"Is that enough?"
"Seven would have been entirely adequate. Seven seconds, in most cases."
Wikipedia defines front-running thus:
Front running, also known as tailgating, is the prohibited practice of entering into an equity (stock) trade, option, futures contract, derivative, or security-based swap to capitalize on advance, nonpublic knowledge of a large ("block") pending transaction that will influence the price of the underlying security. ... A front running firm either buys for its own account before filling customer buy orders that drive up the price, or sells for its own account before filling customer sell orders that drive down the price. Front running is prohibited since the front-runner profits from nonpublic information, at the expense of its own customers, the block trade, or the public market.Follow me below the fold for a discussion of why the architecture of cryptocurrencies means that no-one needs Chombo's mysterious skills to front-run the order flow.
Thursday, October 29, 2020
|Hōryū-ji by Nekosuki, CC-BY-SA|
Now, in The Data of Long-lived Institutions on the Long Now Foundation's blog, Alexander Rose refers to Hōryū-ji:
At about 1,400 years old, these are the two oldest continuously standing wooden structures in the world. And they’ve replaced a lot of parts of them. They keep the roofs on them, and even in a totally humid and raining environment, the central timbers of these buildings have stayed true. Interestingly, this temple was also the place where, over a thousand years ago, a Japanese princess had a vision that she needed to send a particular prayer out to the world to make sure that it survived into the future. And so she had, literally, a million wooden pagodas made with the prayer put inside them, and distributed these little pagodas as far and wide as she could. You can still buy these on eBay right now. It’s an early example of the philosophy of “Lots of Copies Keep Stuff Safe” (LOCKSS).Below the fold, more on Rose's interesting post.
Tuesday, October 27, 2020
As one of his Patreons, I've been reading the chapters as he finished them. It isn't a laugh-a-minute read like his first book, but like the first it is a copiously sourced account of incredible hubris. Facebook's hubris led them to believe that they could, in effect, become a sovereign currency issuer like a government, without any of the responsibilities that governments assume when they control a currency. Actual governments looked at this proposal and responded "you have to be kidding". Follow me below the fold for more.
Tuesday, October 6, 2020
- The data structure is often said to provide immutability or to be tamper-proof, but this is wrong. It is made out of bits, and bits can be changed or destroyed. What it actually provides is tamper-evidence, revealing that the data structure has changed.
- If an unauthorized change to the data structure is detected the damage must be repaired. So there must be multiple replicas of the data structure to allow an undamaged replica to be copied to the damaged replica.
- The role of the consensus mechanism is to authorize changes to the data structure, and prevent unauthorized changes. A change is authorized if the consensus of the replicas agrees to it.
Tuesday, September 29, 2020
The fact that software vendors use licensing to disclaim liability for the functioning of their products is at the root of the lack of security in systems. These proposals are plausible but I believe they would either be ineffective or, more likely, actively harmful. There is so much to write about them that they deserve an entire post to themselves.Below the fold is the post they deserve.
Tuesday, September 22, 2020
I spent nearly two decades building and operating in production the LOCKSS system, a small-ish system that was intended, but never quite managed, to be completely decentralized. I agree with Marlinspike's conclusion, and have been writing with this attitude at least 2014's Economies Of Scale In Peer-to-Peer Networks. It is always comforting to find someone coming to the same conclusion via a completely different route, as with scalability expert Todd Hoff in 2018 and now Moxie Marlinspike based on his experience building the Signal encrypted messaging system. Below the fold I contrast his reasons for skepticism with mine.
Thursday, September 17, 2020
One difficulty was that although academic journals contained some of the Web content that was most important to preserve for the future, the Internet Archive could not access them because they were paywalled. Two years later, Vicky Reich and I started the LOCKSS (Lots Of Copies Keep Stuff Safe) program to address this problem. In 2000's Permanent Web Publishing we wrote:
Librarians have a well-founded confidence in their ability to provide their readers with access to material published on paper, even if it is centuries old. Preservation is a by-product of the need to scatter copies around to provide access. Librarians have an equally well-founded skepticism about their ability to do the same for material published in electronic form. Preservation is totally at the whim of the publisher.Now, Jeffrey Brainard's Dozens of scientific journals have vanished from the internet, and no one preserved them and Diana Kwon's More than 100 scientific journals have disappeared from the Internet draw attention to this long-standing problem. Below the fold I discuss the paper behind the Science and Nature articles.
A subscription to a paper journal provides the library with an archival copy of the content. Subscribing to a Web journal rents access to the publisher's copy. The publisher may promise "perpetual access", but there is no business model to support the promise. Recent events have demonstrated that major journals may vanish from the Web at a few months notice.
This poses a problem for librarians, who subscribe to these journals in order to provide both current and future readers with access to the material. Current readers need the Web editions. Future readers need paper; there is no other way to be sure the material will survive.
Thursday, September 10, 2020
He shows how Amazon's strategy is not to generate and distribute profits, but to re-invest their cash flow into staring and developing businesses. Starting each business absorbs cash, but as they develop they turn around and start generating cash that can be used to start the next one.He is now back with a similarly insightful analysis entitled Amazon's profits, AWS and advertising, which starts:
People argue about Amazon a lot, and one of the most common and long-running arguments is about profits. The sales keep going up, and it takes a larger and larger share of US retail every year (7-8% in 2019), but it never seems to make any money. What’s going on?Below the fold, some details of Evans' explanation.
Tuesday, September 8, 2020
Tuesday, September 1, 2020
I've mentioned before that my father spent his whole career, apart from WW2 as an RNVR watch officer on convoy escorts, at Harrods, the iconic London department store. He even published a textbook on retail distribution. So I can't resist a shout-out to the amazing work of Eric Hutton and the volunteers of Project Gutenberg who, over the last 13 years, have scanned, OCR-ed and proof-read the entire Harrods catalog from 1912. Below the fold, the details.
Thursday, August 27, 2020
Now, Yves Smith's Fed Economists Finger Monopoly Concentration as Underlying Driver of Neoliberal Economic Restructuring; Barry Lynn in Harpers and Fortnite Lawsuit Put Hot Light on Tech Monopoly Power covers three developments in the emerging anti-monopoly consensus:
- Apple and Google ganging up on Epic Games.
- Lina M. Khan's ex-boss Barry Lynn's The Big Tech Extortion Racket: How Google, Amazon, and Facebook control our lives.
- Market Power, Inequality, and Financial Instability by Fed economists Isabel Cairó and Jae Sim
they developed a model to simulate the impact of companies’ rising market power, in conjunction with the assumption that the owners of capital liked to hold financial assets (here, bonds) as a sign of social status. They wanted to see it it would explain six developments over the last forty years. ... And it did!Follow me below the fold for the details.
Thursday, August 20, 2020
Surprisingly, I'm getting good data from CD-Rs more than 14 years old, and from DVD-Rs nearly 12 years old. Your mileage may vary.A year ago I repeated the mind-numbing process of feeding 45 disks through the reader and verifying their checksums. It is time again for this annual chore, and once again this year I failed to find any errors. Below the fold, the details.
Tuesday, August 18, 2020
The Atlantic Council has released a report that looks at the history of computer supply chain attacks.The Atlantic Council also has a summary of the report entitled Breaking trust: Shades of crisis across an insecure software supply chain:
Software supply chain security remains an under-appreciated domain of national security policymaking. Working to improve the security of software supporting private sector enterprise as well as sensitive Defense and Intelligence organizations requires more coherent policy response together industry and open source communities. This report profiles 115 attacks and disclosures against the software supply chain from the past decade to highlight the need for action and presents recommendations to both raise the cost of these attacks and limit their harm.Below the fold, some commentary on the report and more recent attacks.
Tuesday, August 11, 2020
- Two successive successful 51% attacks on Ethereum Classic.
- A new, more realistic estimate of Bitcoin's energy usage; it is only as much as Belgium
Tuesday, August 4, 2020
There is no reason to fear that sites cannot still make money with advertising. That’s because there are already two kinds of highly profitable online ads: contextual ads, based on the content being shown on screen, and behavioral ads, based on personal data collected about the person viewing the ad. Behavioral ads work by tracking your online behavior and compiling a profile about you using your internet activities (and even your offline activities in some cases) to send you targeted ads.He argues that the creepiness of behavioral ads isn't necessary for sites to make money from ads. Below the fold I look at the evidence that Weinberg is right.
Tuesday, July 28, 2020
I started expressing my gradually increasing skepticism the following year. Now, nearly eleven years after Dave's talk, it is time to follow me below the fold for another update.
Tuesday, July 21, 2020
It was about 4 in the afternoon on Wednesday on the East Coast when chaos struck online. Dozens of the biggest names in America — including Joseph R. Biden Jr., Barack Obama, Kanye West, Bill Gates and Elon Musk — posted similar messages on Twitter: Send Bitcoin and the famous people would send back double your money.Two days later Nathaniel Popper and Kate Conger's Hackers Tell the Story of the Twitter Attack From the Inside was based on interviews with some of the perpetrators:
Mr. O'Connor said other hackers had informed him that Kirk got access to the Twitter credentials when he found a way into Twitter’s internal Slack messaging channel and saw them posted there, along with a service that gave him access to the company’s servers. People investigating the case said that was consistent with what they had learned so far. A Twitter spokesman declined to comment, citing the active investigation.Below the fold, some commentary on this and other stories of the fiasco.
Thursday, July 9, 2020
All over this blog (e.g. here) you will find references to W. Brian Arthur's Increasing Returns and Path Dependence in the Economy because it pointed out the driving forces, often called network effects, that cause technology markets to be dominated by one, or at most a few, large players. This is a problem for digital preservation, and for society in general, for both economic and technical reasons. The economic reason is that these natural but unregulated monopolies extract rents from their customers. The technical reason is that they make the systems upon which society depends brittle, subject to sudden, catastrophic and hard-to-recover-from failures.Now, the pandemic has inspired two writers to address the bigger version of the same problem, Bruce Schneier in The Security Value of Inefficiency and Jonathan Aldred in This pandemic has exposed the uselessness of orthodox economics. Below the fold, some commentary.
Tuesday, June 30, 2020
The image is Bill's card from the deck of playing cards the Usenix Association created for the 25th anniversary of the Unix operating system in 1994.
Thursday, June 25, 2020
Our findings on the estimated revenue from transaction fees are in line with the widespread opinion that participation is economically irrational for the majority of the large routing nodes who currently hold the network together. Either traffic or transaction fees must increase by orders of magnitude to make payment routing economically viable.Below the fold I comment on their latest work.
Thursday, June 18, 2020
Two major study retractions in one month have left researchers wondering if the peer review process is broken.Below the fold I explain that the researchers who are only now "wondering if the peer review process is broken" must have been asleep for more than the last decade.
Tuesday, June 16, 2020
open-source software is fully integrated into Google’s Android phones. The volunteer labor of thousands thus helps power Google’s surveillance-capitalist machine.Below the fold, I discuss "the volunteer labor of thousands".
Thursday, June 4, 2020
- Rapid but roughly linear growth in the number of "reliable" journals launched each year. About three times as many were launched in 2018 as in 1978.
- Explosive growth since 2010 in the number of "predatory" journals launched each year. In 2018 almost half of all journals launched were predatory.
Tuesday, June 2, 2020
Below the fold, some commentary on her fascinating article.
Tuesday, May 19, 2020
who hasn’t finished a non-fiction book and thought “Gee, that could have been half the length and just as informative. If that.”Arora et al argue that a cause of the decline in productivity is that:
Yet every now and then you read something that provokes the exact opposite feeling. Where all you can do after reading a tweet, or an article, is type the subject into Google and hope there’s more material out there waiting to be read.
So it was with Alphaville this Tuesday afternoon reading a research paper from last year entitled The changing structure of American innovation: Some cautionary remarks for economic growth by Arora, Belenzon, Patacconi and Suh (h/t to KPMG’s Ben Southwood, who highlighted it on Twitter).
The exhaustive work of the Duke University and UEA academics traces the roots of American academia through the golden age of corporate-driven research, which roughly encompasses the postwar period up to Ronald Reagan’s presidency, before its steady decline up to the present day.
The past three decades have been marked by a growing division of labor between universities focusing on research and large corporations focusing on development. Knowledge produced by universities is not often in a form that can be readily digested and turned into new goods and services. Small firms and university technology transfer offices cannot fully substitute for corporate research, which had integrated multiple disciplines at the scale required to solve significant technical problems.As someone with many friends who worked at the legendary corporate research labs of the past, including Bell Labs and Xerox PARC, and who myself worked at Sun Microsystems' research lab, this is personal. Below the fold I add my 2c-worth to Arora et al's extraordinarily interesting article.
Friday, May 15, 2020
Tuesday, May 5, 2020
A narrowly divided US Supreme Court on Monday upheld the right to freely share the official law code of Georgia. The state claimed to own the copyright for the Official Code of Georgia Annotated and sued a nonprofit called Public.Resource.Org for publishing it online. Monday's ruling is not only a victory for the open-government group, it's an important precedent that will help secure the right to publish other legally significant public documents.Below the fold, commentary on various reports of the decision, and more.
"Officials empowered to speak with the force of law cannot be the authors of—and therefore cannot copyright—the works they create in the course of their official duties," wrote Chief Justice John Roberts in an opinion that was joined by four other justices on the nine-member court.
Tuesday, April 28, 2020
"Smart contracts" are programs, and programs have bugs. Some of the bugs are exploitable vulnerabilities. Research has shown that the rate at which vulnerabilities in programs are discovered increases with the age of the program. The problems caused by making vulnerable software immutable were revealed by the first major "smart contract". The Decentralized Autonomous Organization (The DAO) was released on 30th April 2016, but on 27th May 2016 Dino Mark, Vlad Zamfir, and Emin Gün Sirer posted A Call for a Temporary Moratorium on The DAO, pointing out some of its vulnerabilities; it was ignored. Three weeks later, when The DAO contained about 10% of all the Ether in circulation, a combination of these vulnerabilities was used to steal its contents.
|$25M goes Poof!|
!! ALERT A typo has been found in the code. Because of that, liquidity in expired options contracts can’t be unlocked for new options. !! Please EXERCISE ALL OF YOUR ACTIVE OPTIONS CONTRACTS NOW.Below the fold, some details.
Thursday, April 23, 2020
The virus is reminding us that the purpose of scholarly communication is not to allocate credit for career advancement, and neither is it to keep publishers afloat. Scholarly communication is about, well, scholars communicating with each other, to share insights for the benefit of humanity. And whilst we’ve heard all this before, in a time of crisis we realise afresh that this isn’t just rhetoric, this is reality.Below the fold, a few comments.
Tuesday, April 21, 2020
Records kept since 1940 tell a contrasting story: even as the census has introduced labor-saving technologies, it has required more, not fewer, workers. The efficiency of census-taking appears to have declined over time as it has for most of the economy.Below the fold, some commentary.
Thursday, April 9, 2020
|LoC Web Archive team|
For the past 20 years, a small team of archivists at the Library of Congress has been collecting the web, quietly and dutifully in its way. The initiative was born out of a desire to collect and preserve open-access materials from the web, especially U.S. government content around elections, which makes this the team’s busy season.Kurutz did a good job; the article is well worth reading.
But the project has turned into a sweeping catalog of internet culture, defunct blogs, digital chat rooms, web comics, tweets and most other aspects of online life.
Tuesday, April 7, 2020
From today, Elsevier, a global leader in research publishing and information analytics specializing in science and health, is making all its research and data content on its COVID-19 Information Center available to PubMed Central, the archive of biomedical and lifescience at the US. National Institutes of Health’s National Library of Medicine, and other publicly funded repositories globally, such as the WHO COVID database, for as long as needed while the public health emergency is ongoing. This additional access allows researchers to use artificial intelligence to keep up with the rapidly growing body of literature and identify trends as countries around the world address this global health crisis.Elsevier and the other oligopoly academic publishers have reacted similarly in earlier virus outbreaks. Prof. John Willinsky pounced on this admission that these companies normal restrictive access policies based on copyright ownership slow the progress of science, and thus violate the US Constitution's intellectual property clause:
That Congress shall have Power...To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries.Below the fold I provide some details of his proposal.
Tuesday, March 31, 2020
In Cloud For Preservation I described how much of the motivation for using cloud services was their month-by-month pay-for-what-you-use billing, which transforms capital expenditures (CapEx) into operational expenditures (OpEx). Organizations typically find OpEx much easier to justify than CapEx because:
- The numbers they look at are smaller, even if what they add up to over time is greater.
- OpEx is less of a commitment, since it can be decreased if circumstances change.
Tuesday, March 24, 2020
Tuesday, March 17, 2020
As we see with Bitcoin's Lightning Network, true members of the cryptocurrency cult are not concerned that the foregone interest on capital they devote to making the system work is vastly greater than the fees they receive for doing so. The reason is that, as David Gerard writes, they believe that "number go up". In other words, they are convinced that the finite supply of their favorite coin guarantees that its value will in the future "go to the moon", providing capital gains that vastly outweigh the foregone interest.Follow me below the fold for a discussion of a recent attack on a Proof-of-Stake blockchain that wasn't motivated by the immediate monetary bottom line.
Tuesday, March 10, 2020
Saturday, March 7, 2020
So, below the fold, greatly delayed through my failure, is Michael Nelson's response, which is also available here.
Tuesday, March 3, 2020
In a paper by Nicholas Bloom, Charles Jones and Michael Webb of Stanford University, and John Van Reenen of the Massachusetts Institute of Technology (MIT), the authors note that even as discovery has disappointed, real investment in new ideas has grown by more than 4% per year since the 1930s. Digging into particular targets of research—to increase computer processing power, crop yields and life expectancy—they find that in each case maintaining the pace of innovation takes ever more money and people.Follow me below the fold for some commentary on a number of the other papers they cite.
Thursday, February 27, 2020
Tuesday, February 18, 2020
The project takes two opposite but synergistic approaches:
- Top-Down: Using the bibliographic metadata from sources like CrossRef to ask whether that article is in the Wayback Machine and, if it isn't trying to get it from the live Web. Then, if a copy exists, adding the metadata to an index.
- Bottom-up: Asking whether each of the PDFs in the Wayback Machine is an academic article, and if so extracting the bibliographic metadata and adding it to an index.
Thursday, February 13, 2020
proof-of-work can only achieve payment security if mining income is high, but the transaction market cannot generate an adequate level of income. ... the economic design of the transaction market fails to generate high enough fees.Follow me below the fold for a discussion of a fascinating recent paper that extends Budish's analysis.
Tuesday, February 11, 2020
|Google UI Timeline|
Users complained that Google was trying to trick people into clicking on more paid results, while marketing executives said it was yet another step in blurring the line between ads and unpaid search results, forcing them to spend more money with the internet company.Well, yes, but follow me below the fold for the bigger picture.
Thursday, February 6, 2020
Thursday, January 30, 2020
- How to Regulate (and Not Regulate) Social Media by Jack Balkin
- Bipartisan legislation would force Big Tech to allow interoperability with small competitors by Cory Doctorow
- The Good And The Bad Of The ACCESS Act To Force Open APIs On Big Social Media by Mike Masnick
- Testimony by Maciej Cegłowski to the Senate Committee on Banking, Housing, and Urban Affairs for their hearing on Privacy Rights and Data Collection in a Digital Economy
- A Framework for Regulating Competition on the Internet by Ben Thompson.
- A Better Internet Is Waiting for Us by Annalee Newitz
Tuesday, January 14, 2020
Facebook and Alphabet (Google’s parent), which rely on advertising for, respectively, 97% and 88% of their sales.depend on the idea that targeted advertising, exploiting as much personal information about users as possible, results in enough increased sales to justify its cost.This is despite the fact the both experimental research and the experience of major publishers and advertisers show the opposite. Now, The new dot com bubble is here: it’s called online advertising by Jesse Frederik and Maurits Martijn provides an explanation for this disconnect. Follow me below the fold to find out about it and enjoy some wonderful quotes from them.
Thursday, January 9, 2020
Tuesday, January 7, 2020
Back in June David Gerard asked:
How good a business is running a Lightning Network node? LNBig provides 49.6% ($3.7 million in bitcoins) of the Lightning Network’s total channel liquidity funding — that just sits there, locked in the channels until they’re closed. They see 300 transactions a day, for total earnings on that $3.7 million of … $20 a month. They also spent $1000 in channel-opening fees.Even if the Lightning Network worked (which it doesn't), and were decentralized (which it isn't), Gerard's point was that the transaction fees were woefully inadequate to cover the costs of running a node. Now, A Cryptoeconomic Traffic Analysis of Bitcoin’s Lightning Network by the Hungarian team of Ferenc Béres, István A. Seres, and András A. Benczúr supports Gerard's conclusion with a detailed analysis.
Below the fold, some commentary.
Thursday, January 2, 2020
how we can know that the hardware the software we secured is running on is doing what we expect it to?Bunnie's experience has made him very skeptical of the integrity of the hardware supply chain:
In the process of making chips, I’ve also edited masks for chips; chips are surprisingly malleable, even post tape-out. I’ve also spent a decade wrangling supply chains, dealing with fakes, shoddy workmanship, undisclosed part substitutions – there are so many opportunities and motivations to swap out “good” chips for “bad” ones. Even if a factory could push out a perfectly vetted computer, you’ve got couriers, customs officials, and warehouse workers who can tamper the machine before it reaches the user.Below the fold, some discussion of Bunnie's current project.