The basic aspiration of the LOCKSS Program when we started a quarter century ago was to enable libraries to continue their historical mission of collecting, preserving, and providing readers with access to academic journals. In the paper world libraries which subscribed to a journal owned a copy; in the digital world they could only rent access to the publisher's copy. This allowed the oligoply academic publishers to increase their rent extraction from research and education budgets.
LOCKSS provided a cheap way for libraries to collect, preserve and provide access to their own copy of journals. The competing e-journal preservation systems accepted the idea of rental; they provided an alternate place from which access could be rented if it were denied by the publisher.
Similarly, libraries that purchased a paper book owned a copy that they could loan to readers. The transition to e-books meant that they were only able to rent access to the publisher's copy, and over time the terms of this rental grew more and more onerous.
Below the fold I look into a recent effort to mitigate this problem.
I'm David Rosenthal, and this is a place to discuss the work I'm doing in Digital Preservation.
Tuesday, September 3, 2024
Tuesday, August 27, 2024
2024 Optical Media Durability Update
Six years ago I posted Optical Media Durability and discovered:
It is time once again for the mind-numbing process of feeding 45 disks through the readers to verify their checksums, and yet again this year every single MD5 was successfully verified. Below the fold, the details.
Surprisingly, I'm getting good data from CD-Rs more than 14 years old, and from DVD-Rs nearly 12 years old. Your mileage may vary.Here are the subsequent annual updates:
It is time once again for the mind-numbing process of feeding 45 disks through the readers to verify their checksums, and yet again this year every single MD5 was successfully verified. Below the fold, the details.
Tuesday, August 20, 2024
Astroturfing
I seem to be stuck on the theme of cryptocurrency gaslighting with two weeks ago More Cryptocurrency Gaslighting and one week ago Greenwashing. Now I look at cryptocurrency gaslighting in the political arena, where it is termed astroturfing:
Currently, the crypto-bros have poured money into primaries, defeated several incumbents deemed to be insufficiently crypto-friendly, and have accumulated an immense war-chest for November's general election. This pot of gold was enough to turn Trump from crypto-skeptic to telling Maria Bartiromo:
it is defined as the process of seeking electoral victory or legislative relief for grievances by helping political actors find and mobilize a sympathetic public, and is designed to create the image of public consensus where there is none. Astroturfing is the use of fake grassroots efforts that primarily focus on influencing public opinion and typically are funded by corporations and political entities to form opinions.
Donald Trump, 2019 |
Who knows, maybe we’ll pay off our $35 trillion dollar [national debt], hand them a little crypto check, right? We’ll hand them a little Bitcoin and wipe away our $35 trillionBelow the fold I discuss the gaslighting the cryptosphere is using in their massive attempt to purchase "regulatory clarity", and what the scale of this investment suggests about the profits they expect to garner if it succeeds.
Thursday, August 8, 2024
Greenwashing
Source |
Thursday, August 1, 2024
More Cryptocurrency Gaslighting
SEC vs. Consensys |
The SEC is charging Consensys for unauthorized sales of securities through MetaMask Staking and for failure to register as a broker and a dealer while offering crypto trades and staking services through MetaMask Staking and Swaps. The SEC says that Consensys took $250 million in fees as an unregistered broker.Consensys' defense strategy poses potentially serious problems for the concept of open source, because they are gaslighting about the software that is the basis for the SEC's complaint being open source. Were the court to (a) fall for their gaslighting but (b) agree with the SEC's complaint it could provide a basis for imposing liability on open source developers.
MetaMask is Consensys’ main money maker — a popular browser-based wallet that also lets you stake ETH and buy and sell crypto via decentralized exchanges with “swaps.”
I am afraid that the explanation for why this is so is necessarily rather long but I and others think that it needs to be understood. So stock up with supplies for the journey and follow me below the fold.
Thursday, July 25, 2024
Matt Levine Explains Cryptocurrency Markets
Kanav Kariya |
In many cases, the essential attribute of a crypto token is liquidity: What you want, often, is a token that trades a lot, because your goal for the token is to trade it a lot. Real-world utility, a sensible business model, acceptance in real transactions, etc., are all less important than just trading if you think of crypto as a toy market for traders to play with. If a token trades a lot at a high price, that in itself justifies the price, because that is all that is asked of a token: It doesn’t need to have a good underlying business or cash flows; it just has to trade a lot at a high price.Below the fold I discuss the astonishing story behind this explanation of why wash trading is so rife in cryptocurrencies.
Tuesday, July 16, 2024
Accelerated Computing
Source |
But decades of being congratulated and indulged for the relentless pursuit of their own self-interest has turned the heads of too many of our successful rich. They really believe that they are different: that they owe little to the society from which they have sprung and in which they trade, that taxes are for little people. We are lucky to have them, and, if anything, owe them a favour.Below the fold I continue the "Old man yells at cloud" theme of recent posts by trying to clarify an aspect of the current Jensen Huang hagiography.
Thursday, July 11, 2024
More On The Halvening
At the end of May I wrote One Heck Of A Halvening about the aftermath of the halving of Bitcoin's block reward on April 19th. Six weeks later it is time for a quick update, so follow me below the fold.
Tuesday, July 9, 2024
Engineering For The Long Term
Content Warning: this post contains blatant self-promotion.
Contributions to engineering fields can only reasonably be assessed in hindsight, by looking at how they survived exposure to the real world over the long term. Four of my contributions to various systems have stood the test of time. Below the fold, I blow my own horn four times.
Tuesday, July 2, 2024
X Window System At 40
X11R1 on Sun Techfury90 CC0 |
From: rws@mit-bold (Robert W. Scheifler) To: window@athena Subject: window system X Date: 19 June 1984 0907-EDT (Tuesday) I've spent the last couple weeks writing a window system for the VS100. I stole a fair amount of code from W, surrounded it with an asynchronous rather than a synchronous interface, and called it X. Overall performance appears to be about twice that of W. The code seems fairly solid at this point, although there are still some deficiencies to be fixed up. We at LCS have stopped using W, and are now actively building applications on X. Anyone else using W should seriously consider switching. This is not the ultimate window system, but I believe it is a good starting point for experimentation. Right at the moment there is a CLU (and an Argus) interface to X; a C interface is in the works. The three existing applications are a text editor (TED), an Argus I/O interface, and a primitive window manager. There is no documentation yet; anyone crazy enough to volunteer? I may get around to it eventually. Anyone interested in seeing a demo can drop by NE43-531, although you may want to call 3-1945 first. Anyone who wants the code can come by with a tape. Anyone interested in hacking deficiencies, feel free to get in touch.Scheifler was right that it was a "good starting point for experimentation", but it wasn't really a usable window system until version 11 was released on 15th September 1987. I was part of the team that burned the midnight oil at MIT to get that release out, but my involvement started in late 1985.
Below the fold are some reflections on my contributions, some thoughts on the astonishing fact that the code is still widely deployed after 40 years, and some ideas on why it has been so hard to replace.
Thursday, June 20, 2024
Tuesday, June 11, 2024
Video Game Preservation
Source |
The largest major platform shutdown in recent memory is the closure of the digital stores for the Nintendo 3DS and Wii U platforms. Nintendo shut down the 3DS and Wii U eShops on March 27, 2023, resulting in the removal of 2,413 digital titles. Although many of these are likely available on other platforms, Video Games Chronicle estimates that over 1,000 of those games were exclusive to those platforms’ digital stores and are no longer available in any form, including first-party Nintendo titles like Dr. Luigi, Dillon’s Rolling Western, Mario & Donkey Kong: Minis on the Move, and Pokémon Rumble U. The closures also affected around 500 historical games reissued by Nintendo through their Virtual Console storefronts, over 300 of which are believed not available on any other platform or service.Below the fold I discuss recent developments in this area.
Thursday, June 6, 2024
The Great MEV Heist
The Department of Justice indicted two brothers for exploiting mechanisms supporting Ethereum's "Maximal Extractable Value" (MEV). Ashley Berlanger's MIT students stole $25M in seconds by exploiting ETH blockchain bug, DOJ says explains:
Anton, 24, and James Peraire-Bueno, 28, were arrested Tuesday, charged with conspiracy to commit wire fraud, wire fraud, and conspiracy to commit money laundering. Each brother faces "a maximum penalty of 20 years in prison for each count," the DOJ said.Below the fold I look into the details of the exploit as alleged in the indictment, and what it suggests about the evolution of Ethereum.
The alleged scheme was launched in December 2022 by the brothers, who studied at MIT, after months of planning, the indictment said. The pair seemingly relied on their "specialized skills" and expertise in crypto trading to fraudulently gain access to "pending private transactions" on the blockchain, then "used that access to alter certain transactions and obtain their victims’ cryptocurrency," the DOJ said
Thursday, May 30, 2024
One Heck Of A Halvening
The fundamental idea behind Bitcoin is that, if you restrict the supply of something, its price will rise. That is why the system arranges that there will only ever be 21 million Bitcoin by halving the reward paid for mining the next block every 210,000 blocks (about every four years), an event called the "halvening" (or more recently just the halving). It is an article of faith among the crypto-bros that, after the halvening, the price will rise. For example:
In the image below, the vertical blue lines indicate the previous three halvings (2012-11-28, 2016-7-9, and 2020-5-11). Note how the price has jumped significantly after each halving.The most recent halvening happened on Friday, April 19th. It was eagerly awaited so, six weeks later, it is time to go below the fold and look at the effects.
Source
Tuesday, May 28, 2024
Library of Congress: Designing Storage Architectures 2024
I participated virtually in the 2024 Library of Congress Designing Storage Architectures for Digital Collections meeting. As usual, there were a set of very interesting talks. The slides from the presentations are now online so, below the fold, I discuss the talks I found particulary interesting.
Thursday, May 23, 2024
"Sufficiently Decentralized"
Mining Pools 5/17/24 |
when I look at Bitcoin today, I do not see a central third party whose efforts are a key determining factor in the enterprise. The network on which Bitcoin functions is operational and appears to have been decentralized for some time, perhaps from inception.Below the fold, thanks to a tip from Molly White, I look at recent research suggesting that there is in fact a "central third party" coordinating the enterprise of Bitcoin mining.
...
Over time, there may be other sufficiently decentralized networks and systems
Tuesday, May 21, 2024
Pew Research On Link Rot
Source |
we collected a random sample of just under 1 million webpages from the archives of Common Crawl, an internet archive service that periodically collects snapshots of the internet as it exists at different points in time. We sampled pages collected by Common Crawl each year from 2013 through 2023 (approximately 90,000 pages per year) and checked to see if those pages still exist today.Their results are not surprising, but there are a number of surprising things about their report. Below the fold, I explain.
We found that 25% of all the pages we collected from 2013 through 2023 were no longer accessible as of October 2023. This figure is the sum of two different types of broken pages: 16% of pages are individually inaccessible but come from an otherwise functional root-level domain; the other 9% are inaccessible because their entire root domain is no longer functional.
Thursday, May 16, 2024
Fee-Only Bitcoin
Mining a Bitcoin block needs to be costly to ensure that the gains from an attack on the blockchain are less than the cost of mounting it. Miners have two sources of income to defray their costs, the block rewards and the fees for the transactions in the block.
On April 19th the block reward was halved from 6.25BTC to 3.125BTC. This process is repeated every 210,000 blocks (about every 4 years). It limits the issuance of BTC to 21M because around 2140 the reward will be zero; a halving will make it less than a satoshi.
Long before 2140 the block rewards will have shrunk to become insignificant compared to the fees. Below the fold I look at the significance of the change to a fee-only Bitcoin
On April 19th the block reward was halved from 6.25BTC to 3.125BTC. This process is repeated every 210,000 blocks (about every 4 years). It limits the issuance of BTC to 21M because around 2140 the reward will be zero; a halving will make it less than a satoshi.
Long before 2140 the block rewards will have shrunk to become insignificant compared to the fees. Below the fold I look at the significance of the change to a fee-only Bitcoin
Thursday, May 9, 2024
Elon Musk: Threat Or Menace? Part 5
Source |
- A driver who believed Autopilot could drive him home despite his being drunk. The car drove the wrong way on the highway and killed another innocent victim of Musk's hype.
- Autopilot rear-ending a merging vehicle and killing another innocent victim, a 15-year-old.
- Autopilot slamming into a broken down vehicle on the highway. When the Tesla driver left the wreck she was hit and killed by another car.
- Autopilot speeding through a T-junction and crashing into a parked truck.
"If somebody doesn’t believe Tesla is going to solve autonomy, I think they should not be an investor in the company."Elon Musk, 24th April 2024
Tuesday, April 30, 2024
The 50th Asilomar Microcomputer Workshop
Source |
Brian Berg has written a short history of the workshop for the IEEE entitled The Asilomar Microcomputer Workshop: Its Origin Story, and Beyond. It was started by "Three Freds and a Ted" and one of the Freds, Fred Coury has also written about it in here.Six years ago David Laws wrote The Asilomar Microcomputer Workshop and the Billion Dollar Toilet Seat for the Computer History Museum.
I have attended almost all of them since 1987. I have been part of the volunteer crew for many, including this one, and have served on the board of the 501C3 behind the workshop for some years.
This year's program featured a keynote from Yale Patt, and a session from four of his ex-students, Michael Shebanow, Wen-mei Hwu, Onur Mutlu and Wen-Ti Liu. Other talks came from Alvy Ray Smith based on his book A Biography of the Pixel, Mary Lou Jepsen on OpenWater, her attempt to cost-reduce diagnosis and treatment, and Brandon Holland and Jaden Cohen, two high-school students on applying AI to the Prisoner's Dilemma. I interviewed Chris Malachowsky about the history of NVIDIA. And, as always, the RATS (Rich Asilomar Tradition Session) in which almost everyone gives a 10-minute talk lasted past midnight.
The workshop is strictly off-the-record unless the speaker publishes it elsewhere, so I can't discuss the content of the talks.
Tuesday, April 23, 2024
The Roach Motel Of Banking
Source |
Follow me below the fold as I discuss the article and finally get to why the sting in the tail is interesting..
Tuesday, April 16, 2024
Elon Musk: Threat or Menace Part 4
The previous post in this series, Elon Musk: Threat or Menace Part 3, was based on the impressively detailed reporting from a team at the Washington Post on the crash that killed Jeremy Banner in The final 11 seconds of a fatal Tesla Autopilot crash. The team's subsequent equally detailed Tesla worker killed in fiery crash may be first ‘Full Self-Driving’ fatality triggered this comment which concluded:
It seems the driver thought that it was OK to drive home with a blood alcohol level of 0.26 because he believed Musk's hype that Fake Self Driving would handle it despite having to repeatedly override it on the way out.Now, the team's Faiz Siddiqui and Trisha Thadani are out with In 2018 crash, Tesla’s Autopilot just followed the lane lines. Below the fold I look into what it reveals about Autopilot.
Friday, April 12, 2024
Decentralized Systems Aren't
Below the fold is the text of a talk I gave to Berkeley's Information Systems Seminar exploring the history of attempts to build decentralized systems and why so many of them end up centralized.
Tuesday, April 9, 2024
The Little Garden
Source |
The reason I'm now able to tell this story is that Tom Jennings, the moving spirit behind the ISP has two posts describing the history of The Little Garden, which was the name the ISP had adopted (from a Chinese restaurant in Palo Alto) when I joined it in May 1993. Tom's perspective from the ISP's point of view contrasts with my perspective — that of a fairly early customer enhanced by information via e-mail from John Gilmore and Tim Pozar, who were both involved far earlier than I.
Tuesday, April 2, 2024
The Left Curve
@tzedonn |
There is a surprising amount of respect for people who appear to know nothing about the industry. They’re known as the “left curves.”Below the fold I look at the left side of the curve
The nickname comes from a popular meme in crypto that shows a bell curve with investors on the left who know nothing, or very little, and those in the fat middle of the curve who know something about crypto. On the right are investors who seemingly know everything.
Tuesday, March 19, 2024
More On Pig Butchering
Thankfully, pig butchering scams are getting attention. Three weeks after I posted Tracing The Pig Butchers, John M. Griffin and Kevin Mei posted How Do Crypto Flows Finance Slavery? The Economics of Pig Butchering:
Their Figure 9 shows the flow of funds over time into the scammer's wallets at exchanges. This is how they estimated the $75.3B; their extremely conservative estimate is $35.1B, and their liberal estimate is $237.6B. Note the huge ~$45B increase from January 2021 to January 2023, partly driven by the cryptocurrency boom, and the slowing until January 2024. Presumably the ETF pump will accelerate the rate.
Below the fold, some commentary on this and other recent developments.
Through blockchain addresses used by ‘‘pig butchering’’ victims, we trace crypto flows and uncover methods commonly used by scammers to obfuscate their activities, including multiple transactions, swapping between cryptocurrencies through DeFi smart contracts, and bridging across blockchains. The perpetrators interact freely with major crypto exchanges, sending over 104,000 small potential inducement payments to build trust with victims. Funds exit the crypto network in large quantities, mostly in Tether, through less transparent but large exchanges—Binance, Huobi, and OKX. These criminal enterprises pay approximately 87 basis points in transaction fees and appear to have recently moved at least $75.3 billion into suspicious exchange deposit accounts, including $15.2 billion from exchanges commonly used by U.S. investors. Our findings highlight how the ‘‘reputable’’ crypto industry provides the common gateways and exit points for massive amounts of criminal capital flows. We hope these findings will help shed light on and ultimately stop these heinous crimes.
Griffin & Wei Fig. 9 |
Below the fold, some commentary on this and other recent developments.
Tuesday, March 12, 2024
Petabit Optical Media?
Source |
we increase the capacity of [optical data storage] to the petabit level by extending the planar recording architecture to three dimensions with hundreds of layers, meanwhile breaking the optical diffraction limit barrier of the recorded spots. We develop an optical recording medium based on a photoresist film doped with aggregation-induced emission dye, which can be optically stimulated by femtosecond laser beams. This film is highly transparent and uniform, and the aggregation-induced emission phenomenon provides the storage mechanism. It can also be inhibited by another deactivating beam, resulting in a recording spot with a super-resolution scale. This technology makes it possible to achieve exabit-level storage by stacking nanoscale disks into arrays, which is essential in big data centres with limited space.Below the fold I discuss this technology.
Tuesday, March 5, 2024
Microsoft's Archival Storage Research
Exabytes | Revenue | $/GB | |
---|---|---|---|
Flash | 120 | $38.7B | $0.320 |
Hard Disk | 693 | $26.8B | $0.039 |
LTO Tape | 40 | $0.65B | $0.016 |
- The value that can be extracted from data decays rapidly with time.
- Thus companies would rather invest in current than archival data.
- Thus archival media and systems are a niche market.
- Thus archival media and systems lack the manufacturing volume to drive down prices.
- Thus although quasi-immortal media have low opex (running cost), they have high capex (purchase cost).
- Especially now interest rates are non-zero, the high capex makes the net present value of their lifetime cost high.
- Archival media compete with legacy generic media, which have mass-market volumes and have already amortized their R&D, so have low capex but higher opex through their shorter service lives.
- Because they have much higher volumes and thus much more R&D, generic media have much higher Kryder rates, meaning that although they need to be replaced over time, each new unit at approximately equal cost replaces several old units, reducing opex.
- Especially now interest rates are non-zero, the net present value of the lower capex but higher opex is likely very competitive.
Thursday, February 22, 2024
Competition-proofing
Source |
I've written several times about how in pursuit of a quicker buck, VCs have largely discarded the slow process of building an IPO-ready company like Nvidia in favor of building one that will be acquired by one of the dominant monopolists. These VCs don't support innovation. Even if their acquisition-bound companies do innovate in their short lives, their innovations are rarely tested in the market after the acuisition.
Below the fold I discuss a new paper that presents a highly detailed look at the mechanisms the dominant companies use to neutralize the threats startups could pose to their dominance.
Tuesday, February 13, 2024
Clouds Over The Mines
In early December 2022 when I wrote skeptically about the economics of Bitcoin mining in Foolish Lenders the Bitcoin "price" was around $17K. It has now climbed 153% to around $43K and, below the fold, I am still posting skeptically about the economics of mining.
Thursday, February 8, 2024
Tracing The Pig Butchers
"Vicky" |
I showed my phone to my friend and explained that I was stringing Vicky along because I’d heard about a new kind of investment fraud that often started with a random text message. I had a hunch that this was why “Vicky” was texting me. The scam was called “pig butchering” because the scammers liked to build up the victim’s confidence with a pretend romantic relationship and made-up investment gains before stealing all their money in one fell swoop—like how hogs are fattened up before their slaughter.This is a romance- and cryptocurrency-enabled version of the "Wee Forest Folk" scam we described in our 2003 SOSP paper.
Below the fold, I look into the details of pig-butchering scams, and how the tracing techniques I discussed in Criming On The Blockchain are being applied to identify the cryptocurrency companies facilitating it.
Thursday, February 1, 2024
The Stanford Digital Library Project
The Stanford Digital Library Project stated its goal thus:
In particular Vicky explained citation indices, the concept behind Page Rank, to Larry Page and Sergey Brin. Andy Bechtolsheim was famously instrumental in persuading them to turn their demo of a Page Rank search engine into Google, the company. In his fascinating interview in the Computer History Museum's oral history collection, Andy explains why the idea of ranking pages by their inbound links was so important.
Below the fold I have taken the liberty of transcribing and cleaning up the relevant section of Andy's stream of conciousness, both because it is important history and because it exactly reflects the Andy I was privileged to know in the early days of Sun Microsystems.
The Stanford Integrated Digital Library Project will develop enabling technologies for an integrated “virtual” library to provide an array of new services and uniform access to networked information collections. The Integrated Digital Library will create a shared environment linking everything from personal information collections, to collections of conventional libraries, to large data collections shared by scientists.Stanford librarians Vicky Reich and Rebecca Wesley provided the "library" input for the research.
Wayback Machine, 11/11/98 |
Below the fold I have taken the liberty of transcribing and cleaning up the relevant section of Andy's stream of conciousness, both because it is important history and because it exactly reflects the Andy I was privileged to know in the early days of Sun Microsystems.
Tuesday, January 30, 2024
Criming On The Blockchain
I apologize for the delay in posting but, as you will see, the post I was working on grew rather long.
It seems obvious that doing crimes and writing the receipts to an immutable public ledger is risky, but many criminals have been convinced that there is no risk because cryptocurrencies such as Bitcoin are anonymous. Although there are cryptocurrencies with anonymous transactions, such as Monero and zCash, they are much more difficult to use and much less liquid than pseudonymous cryptocurrencies like Bitcoin. As many criminals have discovered, without an unrealistically intense focus on operational security (opsec), the identity behind the pseudonym can be revealed. An entire industry has evolved to do these revelations, tracing the flow of coins through their blockchains.
Below the fold I discuss the techniques and results of blockchain tracing, based on four main sources:
It seems obvious that doing crimes and writing the receipts to an immutable public ledger is risky, but many criminals have been convinced that there is no risk because cryptocurrencies such as Bitcoin are anonymous. Although there are cryptocurrencies with anonymous transactions, such as Monero and zCash, they are much more difficult to use and much less liquid than pseudonymous cryptocurrencies like Bitcoin. As many criminals have discovered, without an unrealistically intense focus on operational security (opsec), the identity behind the pseudonym can be revealed. An entire industry has evolved to do these revelations, tracing the flow of coins through their blockchains.
Below the fold I discuss the techniques and results of blockchain tracing, based on four main sources:
- Andy Greenberg's new book entitled Tracers in the Dark: The Global Hunt for the Crime Lords of Cryptocurrency.
- Chainalysis' report 2024 Crypto Crime Trends: Illicit Activity Down as Scamming and Stolen Funds Fall, But Ransomware and Darknet Markets See Growth and his ‘Stablecoins’ Enabled $40 Billion in Crypto Crime Since 2022.
- The UN Office for Drugs and Crime (UNODC) report Casinos, Money Laundering, Underground Banking, and Transnational Organized Crime in East and Southeast Asia: A Hidden, Accelerating Threat.
- Recent posts on ChainArgos blog.
Thursday, January 18, 2024
A Lesson Learned
You know how backups work great until you really need them? Below the fold, a lesson learned from my recent example of this phenomenon.
Tuesday, January 9, 2024
Autonomous Vehicles: Trough of Disillusionment
Jeremykemp CC BY-SA 3.0, Link |
Below the fold I try to catch up with the flood of reporting on the autonomous vehicle winter triggered by the bursting of the bubble.
Tuesday, January 2, 2024
Good News For Tether
USDT "market cap" |
BTC "price" |
So all is well with the world; Tether gets to keep the interest on another $20B, which at say 4% is an extra $800M/year on their bottom line, and the Bitcoin HODL-ers see their
Subscribe to:
Posts (Atom)