Thursday, March 27, 2025

Software Supply Chain Attack

Joel Wallenberg interviewed me on 14th February for his article in the 28th February edition of Grant's Interest Rate Observer entitled Memo to the bitcoiners. Alas, it is paywalled, but among the many quotes from me Wallenberg used was that blockchain-based systems "are very vulnerable to supply-chain attacks".

Exactly a week after the interview and a week before the article went to press, we got an example, the biggest cryptocurrency heist in history. Below the fold I discuss the details.

Among the countries the US recently voted with in the UN General Assembly is the Democratic Peoples' Republic of Korea (DPRK). The US administration may have been impressed with the DPRK's innovative, diversified business model. This includes divisions responsible for supporting the Russian invasion of Ukraine with soldiers and arms, counterfeiting $100 bills, drug smuggling, and trafficking wildlife and humans. One of the most profitable divisions is responsible for stealing cryptocurrencies, and on 21st February it had a major success by stealing Ethereum with a notional value of $1.5B from the Bybit exchange.

The Economist asks Why are North Korean hackers such good crypto-thieves? It is a good question because:
n 2023 North Korean hackers made away with a total of $661m, according to Chainalysis, a crypto-investigations firm; they doubled the sum in 2024, racking up $1.34bn across 47 separate heists, an amount equivalent to more than 60% of the global total of stolen crypto. The ByBit operation indicates a growing degree of skill and ambition: in a single hack, North Korea swiped the equivalent of $1.5bn from the exchange, the largest-ever heist in the history of cryptocurrency.
Source
The reason is that they are a mainstay of the DPRK's economy:
Crypto-thievery is a more efficient way to earn hard currency than traditional sources, such as overseas labourers or illegal drugs. The United Nations Panel of Experts (UNPE), a monitoring body, reported in 2023 that cyber-theft accounted for half of North Korea’s foreign-currency revenue. North Korea’s digital plunder last year was worth more than three times the value of its exports to China, its main trade partner. “You take what took millions of labourers, and you can replicate that with the work of a few dozen people,” says Mr Carlsen.
The attribution of the Bybit theft to the DPRK is convincing, because the modus operandi was very similar to last October's theft of $50M from Radiant Capital's multisig cold wallet. Radiant reported:
Attackers were able to compromise the devices of at least these three core contributors through a sophisticated malware injection. These compromised devices were then used to sign malicious transactions.

Although three compromised devices have been confirmed, it is likely that more were targeted — the means by which they were compromised remains unknown and under investigation. The devices were compromised in such a way that the front-end of Safe{Wallet} (f.k.a. Gnosis Safe) displayed legitimate transaction data while malicious transactions were signed and executed in the background.
This is kind of like Obi-Wan Kenobi's "These aren't the droids you're looking for".

Radiant was being careful. They used Safe{Wallet} to implement a multi-sig wallet, which required at least three signatures to authorize a transaction. And they checked the transactions carefully:
Each transaction was simulated for accuracy on Tenderly and individually reviewed by multiple developers at each signature stage. Front-end checks in both Tenderly and Safe showed no anomalies during these reviews.

To underscore the significance of this point, the compromise was completely undetectable during the manual review of the Gnosis Safe UI and Tenderly simulation stages of the routine transaction.
Source
Safe{Wallet} is an open-source Web app that creates and manages access to multi-sig wallets. In effect it interposes a user interface layer above the Ethereum blockchain. Last May Alex Miguel's Safe Wallet Review 2025: Pros, Cons, & Features complimented the UI:
The UI for a Safe is clean and intuitive. You can browse the assets in the safe easily, as well as check and manage the participating addresses.
The nature of the Ethereum ecosystem means that this user interface has some issues:
Front-end verification of all three multi-signature transactions showed no signs of compromise, aside from Safe App transaction resubmissions due to failures. It is important to highlight that resubmitting Safe transactions due to failures is a common and expected occurrence. Transactions submitted on the Safe front-end can fail due to gas price fluctuations, nonce mismatch, network congestion, insufficient gas limit, smart contract execution errors, token insufficiency, pending transactions, front-end synchronization issues, timeouts, or permission/signature errors in multi-signature setups. As a result, this behavior did not raise immediate suspicion. The malicious actors exploited this normalcy, using the process to collect multiple compromised signatures over several attempts, all while mimicking the appearance of routine transaction failures.
It is important to note that, at the time they were writing, Radiant did not know how the developer's devices were compromised to sign the malicious transactions. They did set out a list of precautions which, I believe, would have prevented the subsequent Bybit heist, in particular:
Take raw transaction data out of your wallet provider when a signature is prompted (e.g., Metamask, Rabby) and plug it into https://etherscan.io/inputdatadecoder. Confirm that the function you are calling, and the ToAddress all match up with the intended behavior. If a device were to be compromised, like in the case above, it would either not decode at all, call a different function than the one you thought you were calling, or it would result in a different owner address than the one you intended.
The most comprehensive analysis of the Bybit heist I found was Cyfrin's The Safe Wallet Hack That Led to Bybit’s $1.4B Heist. In summary, the attackers:
  1. Compromised a developer machine at Safe
  2. Injected malicious JavaScript into a development container
  3. Specifically targeted Bybit exchange to stay undetected longer
  4. Manipulated what Bybit signers saw in the Safe interface
...
This represents a sophisticated supply chain attack rather than a direct compromise of end-user devices.

This attack followed an almost identical pattern to recent attacks on WazirX Exchange and Radiant Capital. This suggests that the same threat actors are repeatedly using this technique successfully.

It's now believed that the Safe UI was the compromise point in all these cases rather than end-user machines, which explains the similar attack patterns.
Because the attackers had access to the Safe developer's machine, they had access to one of Safe's S3 buckets, from which the Safe code running on the machines at Bybit downloaded the malware.

According to Thanh Nguyen of Verichains' preliminary report:
By examining the machines of three Signers from Bybit, malicious JavaScript payload from app.safe.global was discovered in the Google Chrome cache files.

There are two javascript files that were modified: _app-52c9031bfa03da47.js and 6514.b556851795a4cbaa.js.
...
From the Wayback Archive (https://web.archive.org/), we also identified an instance of this malicious JavaScript file dating back to Feb 19, 2025 17:29:05
The malware was planted on 19th February around 17:29 GMT, and removed about 2 minutes after the heist on 21st February at 14:15 GMT.

Later the same day, @zachxbt attributed the heist to the DPRK's Lazarus Group:
At 19:09 UTC today, @zachxbt submitted definitive proof that this attack on Bybit was performed by the LAZARUS GROUP.

His submission included a detailed analysis of test transactions and connected wallets used ahead of the exploit, as well as multiple forensics graphs and timing analyses.

The submission has been shared with the Bybit team in support of their investigation. We wish them all the best.
Cyfrin agrees with Radiant's key precaution:
Bybit’s team trusted what they saw on their screens, and Safe’s engineering team trusted that their systems were secure.

“Don’t trust, verify” has become a blockchain mantra for a reason.

When the signers reviewed the transaction but did not verify the calldata on their physical hardware devices, everything appeared correct. This underscores the critical need for thorough transaction verification beyond what’s displayed on the screen.

To spell it out, the computers showed a spoofed transaction that tricked them, but their wallets showed the malicious transaction. They could have caught this on the hardware wallet, but as of today, calldata can be tricky to verify on a wallet.
It is likely that the Safe developer's machine was compromised via a phishing attack. That was the modus operandi for the Radiant heist:
The attack began on Sept. 11, when a Radiant Capital developer received a Telegram message from someone impersonating a trusted former contractor. According to the message, the contractor was looking for a new job opportunity in smart contract audits. It requested comments on the contractor’s work and provided a link to a compressed PDF detailing their next assignment. The hackers even mimicked the contractor’s legitimate website to add credibility.

The zip file contained a disguised executable named INLETDRIFT. Upon opening, it installed malware on the developer’s macOS device, granting attackers access to the developer’s system. The malware was designed to communicate with a hacker-controlled server.
The security of a multi-billion dollar exchange depended upon the developers at a software supplier not clicking on a link in a Telegram message.

The new US administration's enthusiasm for grifts such as cryptocurrencies means the outlook for the DPRK's cryptocurrency division is rosy:
Tackling the problem requires multilateral efforts across governments and the private sector, but such collaboration has been fraying. Russia used its UN veto to gut the UNPE last year. President Donald Trump’s cuts to American development aid have hit programmes aimed at building cyber-security capacity in vulnerable countries.

By contrast, the North Korean regime is throwing ever more resources at cybercrime. South Korea’s intelligence services reckon its cybercrime force grew from 6,800 people in 2022 to 8,400 last year. As the crypto-industry expands in countries with weaker regulatory oversight, North Korea has an increasingly “rich target environment”, says Abhishek Sharma of the Observer Research Foundation, an Indian think-tank. Last year, Mr Sharma notes, North Korea attacked exchanges based in India and Indonesia.
Update 27th March
© Grant's Interest Rate Observer
Two weeks after the article that quoted me appeared, the front page of Grant's Interest Rate Observer summed the situation up with this pithy cartoon, which I use with permission. The 28th March edition featured another excellent cryptocurrency-focused article entitled The two lives of Donald J. Trump:
Crypto prices are prone to crashes — one could say that crypto exchanges are designed to facilitate them. It would be strange if the immensity of the market caps involved did not introduce a new element of risk into a world that hardly needs more. As for America’s multitasking president, he boasted (through his lawyer) in 2019 that he could commit murder on Fifth Avenue in broad daylight and not be charged with a crime. A conflict of interest is no capital crime, but Trump continues to mix the business of state with the pleasures and temptations of money-making.
The article features extensive quotes from the wonderful Prof. Carol Alexander about the problem of "rehypothecation" in DeFi:
One can borrow up to 78.5% loan-to-value on Aave against the special tokens and make an interest spread of 29 basis points by staking the newly borrowed ether. Starting with 100 ETH, for instance, one could stake them (earning 3%), use the automatically generated liquid staking tokens to borrow 78 more ETH, stake those to earn a net 29 basis points, use the resulting 78 more liquid staking tokens to borrow 78% of 78, i.e., 61, more ether, stake them for another 29 basis points and so on. ... In fact, an enterprising decentralized financier will need to iterate the borrowing and staking process 12 times until he is levered 4.5 times, having borrowed 3.5 ether tokens for each originally laid out, before the resulting digital hoard will even break 4% in yield,
This all sounds great until the price of the collateral drops, perhaps in one of the flash crashes to which cryptocurrencies are subject:
On May 19, 2021, a flash crash in multiple cryptocurrencies briefly knocked more than 25% off the price of bitcoin and 34% off the price of ether before the prices recovered somewhat, all inside of an hour.
In minutes the collateral will be liquidated automatically, and if that doesn't clear the position, so will any other assets the exchange can get its hands on. The article quotes Alexander:
the entire portfolio, not just in ether, but all the other positions, will get taken out by the exchange. The exchange is not supposed to sell these things straight away. They have things called “guarantee funds” or “insurance funds,” which are supposed to absorb the liquidated positions [and hold them]. But they don’t. They dump them into the market. I’ve got all the data to show that.
This is the business into which Trump's World Liberty Financial wants to get:
“With World Liberty Financial [Trump] stands to make billions. . . . They want to provide lending and borrowing services like Aave. Aave is the biggest shadow bank [in decentralized finance],” Alexander explains, representing a market for crypto-token deposits and loans worth $29.1 billion.

2 comments:

Geoff said...

At the end of paragraph 3, should that be "$1.5B"? Otherwise it's inconsistent with the quote from The Economist...

David. said...

Oops, thanks Geoff!