A Local Large Language Model

I wanted to see what all the hype around Large Language Models (LLMs) amounted to, but the privacy terms in the EULA for Bard were completely unacceptable. So when I read Dylan Patel and Afzal Ahmad's We Have No Moat: And neither does OpenAI:

Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months.

I decided to try open source LLMs for myself, since as they run locally the privacy risk is mitigated. Below the fold I tell the story so far; I will update it as I make progress.

Flash Loans

I have been generally skeptical of claims that blockchain technology and cryptocurrencies are major innovations. Back in 2017 Arvind Narayanan and Jeremy Clark published Bitcoin's Academic Pedigree, showing that Satoshi Nakamoto assembled a set of previously published components in a novel way to create Bitcoin. Essentially the only innovation among the components was the Longest Chain Rule.

But, for good or ill, there is at least one genuinely innovative feature of the cryptocurrency ecosystem and in Flash loans, flash attacks, and the future of DeFi Aidan Saggers, Lukas Alemu and Irina Mnohoghitnei of the Bank of England provide an excellent overview of it. They:

analysed the Ethereum blockchain (using Alchemy’s archive node) and gathered every transaction which has utilised the ‘FlashLoan’ smart contract provided by DeFi protocol Aave V1 and V2. The Aave protocol, one of the largest DeFi liquidity providers, popularised flash loans and is often credited with their design. Using this data we were able to gather 60,000 unique transactions from Aave’s flash loan inception through to 2023

Below the fold I discuss their overview and some of the many innovative ways in which flash loans have been used.

Be Careful What You Vote For

One big idea in cryptocurrencies is attempting to achieve decentralization through "governance tokens" whose HODLers can control a Decentralized Autonomous Organization (DAO) by voting upon proposed actions. Of course, this makes it blindingly obvious that the "governance tokens" are securities and thus regulated by the SEC. But even apart from that problem recent events, culminating in "little local difficulties" for Tornado Cash, demonstrate that there are several others.

Below the fold I look at these problems.

Fractional Reserve Crypto-Banking

Source

Michael Lewis is writing an eagerly awaited book on Sam Bankman-Fried and the collapse of FTX, and on his podcast series he has been interviewing sources he used in his research. The latest one is an entertaining interview with Molly White. The best part is their discussion of how the story justifying cryptocurrencies keeps changing.

Nakamoto's goal for Bitcoin was:

What is needed is an electronic payment system based on cryptographic proof instead of trust, allowing any two willing parties to transact directly with each other without the need for a trusted third party.

Below the fold I examine how this earliest cryptocurrency story changed.

Lies, Damned Lies, & A16Z's Statistics

This post is a quick shout-out to two excellent pieces documenting the corruption of the venture capital industry:

• Molly White's Narrative over numbers: Andreessen Horowitz's State of Crypto report
• Matt Levine's Crypto Had Its Bank Runs Too

Below the fold I comment on both.

Flooding The Zone With Shit

Tom Cowap
CC-BY-SA 4.0

Much of the discussion occupying the Web recently has been triggered by the advent of Large Language Models (LLMs). Much of that has been hypeing the vast improvements in human productivity they promise, and glossing over the resulting unemployment among the chattering and coding classes. But the smaller negative coverage, while acknowledging the job losses, has concentrated on the risk of "The Singularity", the idea that these AIs will go HAL 9000 on us, and render humanity obsolete^[0].

My immediate reaction to the news of ChatGPT was to tell friends "at last, we have solved the Fermi Paradox"^[1]. It wasn't that I feared being told "This mission is too important for me to allow you to jeopardize it", but rather that I assumed that civilizations across the galaxy evolved to be able to implement ChatGPT-like systems, which proceeded to irretrievably pollute their information environment, preventing any further progress.

Below the fold I explain why my on-line experience, starting from Usenet in the early 80s, leads me to believe that humanity's existential threat from these AIs comes from Steve Bannon and his ilk flooding the zone with shit^[2].

The Cryptocurrency Use Case

Paul Le Roux
By Farantgh - Own work

A frequently-asked question about cryptocurrencies is "do they have an actual use case apart from speculation?" Jemima Kelly's Using crypto for crime is not a bug — it’s an industry feature starts:

One of the (many) times I have been heckled during a panel on crypto was when I argued that it shouldn’t be thought of as money. The only reason to use it other than for speculation, I said, was to buy drugs on the internet. This was a preposterous idea, the heckler retorted; crypto is used for so much more than that.

and ends:

So in a funny way, my heckler was right: crypto isn’t just used for speculating on and buying drugs on the internet: it’s used for much murkier criminal activities, too.

Below the fold I discuss the details of the "murkier criminal activities".