Miners' Extractable Value

According to the official Ethereum website "Maximal Extractable Value" (MEV) is a feature not a bug. MEV is a consequence of the fact that it is the miners, or rather in almost all cases the mining pools, that decide which transactions, from the public mempool of pending transactions, or from a dark pool, or from the mining pool itself, will be included in the block that they mine, and in what order. The order is especially important in Turing-complete blockchains such as Ethereum; allowing miners to front-run, back-run or sandwich transactions from elsewhere. The profit from doing so is MEV. MEV is being renamed from Miners Extractable Value to Maximal Extractable Value since it turns out that miners are not the only actors who can extract it.

Ethereum mining 11/07/21

In Ethereum, the MEV profit is enhanced because mining is dominated by a very small number of large pools; last November two pools shared a majority of the mining power. Thus there is a high probability that these pools will mine the next block and thus reap the MEV. Note that activities such as front-running are illegal in conventional finance, although high-frequency traders arguably use these techniques.

I wrote about these issues in Ethereum Has Issues, discussing Philip Daian et al's Flash Boys 2.0: Frontrunning in Decentralized Exchanges, Miner Extractable Value, and Consensus Instability and Julien Piet et al's Extracting Godl [sic] from the Salt Mines: Ethereum Miners Extracting Value, but this just scratched the surface. Below the fold I review ten more contributions.

Eskandari et al (9^th April 2019)

Sok: Transparent dishonesty: front-running attacks on blockchain by S. Eskandari, S. Moosavi, and J. Clark was an early investigation of front-running in Ethereum. From their abstract:

We consider front-running to be a course of action where an entity benefits from prior access to privileged market information about upcoming transactions and trades. Front-running has been an issue in financial instrument markets since the 1970s. With the advent of the blockchain technology, front-running has resurfaced in new forms we explore here, instigated by blockchain’s decentralized and transparent nature. In this paper, we draw from a scattered body of knowledge and instances of front-running across the top 25 most active decentral applications (DApps) deployed on Ethereum blockchain. Additionally, we carry out a detailed analysis of Status.im initial coin offering (ICO) and show evidence of abnormal miner’s behavior indicative of front-running token purchases. Finally, we map the proposed solutions to front-running into useful categories.

They define Ethereum's front-running vulnerability thus:

Any user monitoring network transactions (e.g., running a full node) can see unconfirmed transactions. On the Ethereum blockchain, users have to pay for the computations in a small amount of Ether called gas. The price that users pay for transactions, gasPrice, can increase or decrease how quickly miners will execute them and include them within the blocks they mine. A profit-motivated miner who sees identical transactions with different transaction fees will prioritize the transaction that pays a higher gas price due to limited space in the blocks. ... Therefore, any regular user who runs a full-node Ethereum client can front-run pending transactions by sending adaptive transactions with a higher gas price

They divide front-running attacks into three classes:

displacement, insertion, and suppression attacks. In all three cases, Alice is trying to invoke a function on a contract that is in a particular state, and Mallory will try to invoke her own function call on the same contract in the same state before Alice.

In the first type of attack, a displacement attack, it is not important to the adversary for Alice’s function call to run after Mallory runs her function. Alice’s can be orphaned or run with no meaningful effect. Examples of displacement include: Alice trying to register a domain name and Mallory registering it first; Alice trying to submit a bug to receive a bounty and Mallory stealing it and submitting it first; and Alice trying to submit a bid in an auction and Mallory copying it.

In an insertion attack, after Mallory runs her function, the state of the contract is changed and she needs Alice’s original function to run on this modified state. For example, if Alice places a purchase order on a blockchain asset at a higher price than the best offer, Mallory will insert two transactions: she will purchase at the best offer price and then offer the same asset for sale at Alice’s slightly higher purchase price. If Alice’s transaction is then run after, Mallory will profit on the price difference without having to hold the asset.

In a suppression attack, after Mallory runs her function, she tries to delay Alice from running her function. After the delay, she is indifferent to whether Alice’s function runs or not.

They examined a range of Ethereum DApps. Perhaps the most revealing was their study of the Status.im ICO. As usual during times of high demand, the Ethereum network suffered from severe congestion:

During the time frame the ICO was open for participation, there were reports of Ethereum network being unusable and transactions were not confirming. ... there were many transactions sent with a higher gas price to front-run other transactions, however, these transactions were failing due to the restriction in the ICO smart contract to reject transactions with higher than 50 GWei gas price (as a mitigation against front-running).

Eskandari et al Fig 3

And, as usual, Ethereum mining was dominated by a few large pools; in this case Ethermine and F2Pool controlled 49% of the mining power. Eskandari et al discovered that at least one of them was abusing the system by front-running:

F2Pool—an Ethereum mining pool that had around 23% of the mining hash rate at the time —sent 100 Ether to 30 new Ethereum addresses before the Status.im ICO started. When the ICO opened, F2Pool constructed 31 transactions to the ICO smart contract from their addresses, without broadcasting the transactions to the network. They used their entire mining power to mine their own transactions and some other potentially failing high gas price transactions.

The "high gas price transactions" were added to congest the network and reduce the chance that competing transactions would be confirmed. Eskandari et al tracked the Ether from the 30 addresses to discover:

the funds deposited by F2Pool in these addresses were sent to Status.im ICO and mined by F2Pool themselves, where the dynamic ceiling algorithm refunded a portion of the deposited funds. A few days after these funds were sent back to F2Pool main address and the tokens were aggregated later in one single address.

This early example clearly shows a mining pool using their ability to insert non-public transactions to generate MEV.

Zhou et al (29^th September 2020)

In High-frequency trading on decentralized on-chain exchanges by L. Zhou, K. Qin, C. F. Torres, D. V. Le, and A. Gervais:

focus on a combination of front- and back-running, known as a sandwiching, for a single onchain DEX. To the best of our knowledge, we are the first to formalize and quantify sandwich attacks.

They stress that:

While the SEC defines front-running as an action on private information, we only operate on public trade information.

They claim four contributions:

• Formalization of sandwich attacks. We state a mathematical formalization of the AMM [Automated Market Maker] mechanism and the sandwich attack, providing an adversary with a framework to manage their portfolio of assets and maximize the profitability of the attack.
• Analytic and empirical evaluation. We analytically and empirically evaluate sandwich attacks on AMM DEX. Besides an adversarial liquidity taker, we introduce a new class of sandwich attacks performed by an adversarial liquidity provider. We quantify the optimal adversarial revenue and validate our results on the Uniswap exchange (largest DEX, with 5M USD trading volume at the time of writing). Our empirical results show that an adversary can achieve an average daily revenue of 3,414 USD. Even without collusion with a miner, we find that, in the absence of other adversaries, the likelihood to position a transaction before or after another transaction within a blockchain block is at least 79%, using a transaction fee payment strategy of ±1 Wei.
• Multiple Attacker Game. We simulate the sandwich attacks under multiple simultaneous attackers that follow a reactive counter-bidding strategy. We find that the presence of 2, 5 and 10 attackers respectively reduce the expected profitability for each attacker by 51.0%, 81.4% and 91.5% to 0.45, 0.17, 0.08 ETH (67, 25, 12 USD), given a victim that transacts 20 ETH to DAI on Uniswap with a transaction pending on the P2P layer for 10 seconds before being mined. If the blockchain is congested (i.e. the victim transaction remains pending for longer than the average block interval), we show that the breakeven of the attacker becomes harder to attain.
• DEX security vs. scalability tradeoff. Our work uncovers an inherent tension between the security and scalability of an AMM DEX. If the DEX is used securely (i.e. under a low or zero price slippage), trades are likely to fail under high transaction volume; and an adversarial trader may profit otherwise.

There are a number of interesting points here. It is striking that even a small 1 Wei difference gave almost 4 in 5 chance of correct positioning. And that means that even without a miner a lone sandwich trader could have made about $1.25M/year. But, of course, an opportunity like this is rapidly competed away, especially if the victim transaction is visible for more than one block. The advantage of dark pools is clear.

The tradeoff they identify is interesting, because it is based on the inevitable "price slippage":

Price slippage is the change in the price of an asset during a trade. Expected price slippage is the expected increase or decrease in price based on the volume to be traded and the available liquidity, where the expectation is formed at the beginning of the trade. The higher the quantity to be traded, the greater the expected slippage ... Unexpected price slippage refers to any additional increase or decrease in price, over and above the expected slippage, during the interveni[ng] period from the submission of a trade commitment to its execution.

They describe the tradeoff thus:

Our work, sheds light on a dilemma facing DEXs: if the default slippage is set too low, the DEX is not scalable (i.e. only supports few trades per block), if the default slippage is too high, adversaries can profit.

Zhou et al (3^rd March 2021)

In On the just-in-time discovery of profit-generating transactions in defi protocols, Liyi Zhou, Kaihua Qin, Antoine Cully, Benjamin Livshits and Arthur Gervais:

investigate two methods that allow us to automatically create profitable DeFi trades, one well-suited to arbitrage and the other applicable to more complicated settings. We first adopt the Bellman-Ford-Moore algorithm with DEFIPOSERARB and then create logical DeFi protocol models for a theorem prover in DEFIPOSER-SMT. While DEFIPOSER-ARB focuses on DeFi transactions that form a cycle and performs very well for arbitrage, DEFIPOSER-SMT can detect more complicated profitable transactions. We estimate that DEFIPOSER-ARB and DEFIPOSER-SMT can generate an average weekly revenue of 191.48 ETH (76,592 USD) and 72.44 ETH (28,976 USD) respectively, with the highest transaction revenue being 81.31 ETH (32,524 USD) and 22.40 ETH (8,960 USD) respectively.

The key to their analysis is the ability to execute trades on multiple DeFi platforms:

A peculiarity of DeFi platforms is their ability to interoperate; e.g., one may borrow a cryptocurrency asset on one platform, exchange the asset on another, and for instance, lend the resulting asset on a third system. DeFi’s composability has led to the emergence of chained trading and arbitrage opportunities throughout the tightly intertwined DeFi space. Reasoning about what this easy composition entails is not particularly simple; on one side, atomic composition allows to perform risk-free arbitrage — that is to equate asset prices on different DeFi markets. Arbitrage is a benign and important endeavor to keep markets synchronized.

On the other side, we have seen multi-million-revenue trades that cleverly use the technique of flash loans to exploit economic states in DeFi protocols ... While exploiting economic states, however, is not a security attack in the traditional sense, the practitioners’ community often frames these high-revenue trades as “hacks.” Yet, the executing trader follows the rules set forth by the deployed smart contracts. Irrespective of the framing, liquidity providers engaging with DeFi experience millions of USD in unexpected losses.

They validate their approach by showing that:

DEFIPOSER-SMT finds the known economic bZx attack from February 2020, which yields 0.48M USD. Our forensic investigations show that this opportunity existed for 69 days and could have yielded more revenue if exploited one day earlier.

The details of the attack DEFIPOSER-SMT found were:

On the 15th of February, 2020, a trader performed a pump and arbitrage attack on the margin trading platform bZx3 . The core of this trade was a pump and arbitrage involving four DeFi platforms atomically executed in one single transaction. ... this trade resulted in in 4,337.62 ETH (1,735,048 USD) loss from bZx loan providers, where the trader gained 1,193.69 ETH (477,476 USD) in total

Zhou et al Fig 7

One interesting aspect of their results is the relationship between capital employed and return:

We visualize in Figure 7 the revenue generated by DEFIPOSER-SMT and DEFIPOSER-ARB as a function of the initial capital. If a trader owns the base asset (e.g., ETH), most strategies require less than 150 ETH. Only 10 strategies require more than 100 ETH for DEFIPOSER-SMT, and only 7 strategies require more than 150 ETH for DEFIPOSER-ARB.

But this need for capital can be avoided:

This capital requirement is reduced to less than 1.00 ETH (400 USD) when using flash loans (cf. Figure 7 (b, d)).

Eric Budish showed that, to be safe, the value of transactions in a Bitcoin block must not exceed the block reward. Zhou et al study the effect of the MEV they find on Ethereum's safety:

Looking beyond the financial gains mentioned above, forks deteriorate the blockchain consensus security, as they increase the risks of double-spending and selfish mining. We explore the implications of DEFIPOSER-ARB and DEFIPOSER-SMT on blockchain consensus. Specifically, we show that the trades identified by our tools exceed the Ethereum block reward by up to 874×. Given optimal adversarial strategies provided by a Markov Decision Process (MDP), we quantify the value threshold at which a profitable transaction qualifies as Miner Extractable Value (MEV) and would incentivize MEV-aware miners to fork the blockchain. For instance, we find that on Ethereum, a miner with a hash rate of 10% would fork the blockchain if an MEV opportunity exceeds 4× the block reward.

ETH mining 09/03/22

Note that, as I write, three mining pools have more than 10% of the ETH hash rate (Ethermine 27.1, f2pool 15.5, Hiveon.net 10.2) for a total of 52.8%. The risk to Ethereum safety that Zhou et al identify is thus very real.

Torres et al (11^th August 2021)

Frontrunner jones and the raiders of the dark forest: An empirical study of frontrunning on the ethereum blockchain by Christof Ferreira Torres, Ramiro Camino and Radu State:

aims to shed some light into what is known as a dark forest and uncover these predators’ actions. We present a methodology to efficiently measure the three types of frontrunning: displacement, insertion, and suppression. We perform a largescale analysis on more than 11M blocks and identify almost 200K attacks with an accumulated profit of 18.41M USD for the attackers, providing evidence that frontrunning is both, lucrative and a prevalent issue.

That is an average of about $9,200 per attack. Given the skewed distribution, this is surprisingly small. The authors provide a breakdown of attack cost and profit in their Figure 7. The most likely results are:

• Displacement: cost around $10 and profit over $100.
• Insertion: cost below $10 and profit under $100.
• Suppression: cost around $1,000 and profit around $10,000.

Thus the typical return on investment is around 10x. Despite the relatively small dollar profit per attack, automation and the 10x RoI is enough to motivate attackers.

Torres et al observe the same DEX dilemma as Zhou et al:

Uniswap, the DEX most affected by frontrunning, is aware of the frontrunning issue and proposes a slippage tolerance parameter that defines how distant the price of a trade can be before and after execution. The higher the tolerance, the more likely the transaction will go through, but also the easier it will be for an attacker to frontrun the transaction. The lower the tolerance, the more likely the transaction will not go through, but also the more difficult it will be for an attacker to frontrun the transaction. As a result, Uniswap’s users find themselves in a dilemma.

Judmayer et al (21^st September 2021)

Estimating (miner) extractable value is hard, let’s go shopping! by Aljosha Judmayer, Nicholas Stifter, Philipp Schindler, and Edgar Weippl describes:

different forms of extractable value and how they relate to each other. Furthermore, we outline a series of observations which highlight the difficulties in defining these different forms of extractable value and why a generic and thorough definition of it (and thus its precise calculation) is impossible for permissionless cryptocurrencies without assuming bounds regarding all available resources (e.g., other cryptocurrencies) that are, or could be of relevance for economically rational players. We also describe a way to estimate the minimum extractable value, measured in multiples of normalized block rewards of a reference resource, to incentivize adversarial behaviour of participants which can lead to consensus instability. In the end, we propose a peculiar yet straightforward technique for choosing the personal security parameters k regardless of extractable value opportunities.

They explain the role of k:

As the concept of MEV/BEV is tied to the economic incentives of whether or not to fork a certain block/chain [44], this question also relates to economic considerations regarding the choice of the personal security parameter k of merchants. An accurate estimation of the overall MEV/BEV value, would allow to adapt and increase k accordingly in periods of high overall MEV/BEV. The choice of the security parameter k, which determines the number of required confirmation blocks until a payment can safely be considered confirmed, has been studied in a variety of works. Rosenfeld [35] showed that, although waiting for more confirmations exponentially decreases the probability of successful attacks, no amount of confirmations will reduce the success rate of attacks to 0 in the probabilistic security model of PoW, and that there is nothing special about the often-cited figure of k = 6 confirmations.

Goharshady showed that there is inevitable risk for merchants in selling goods (or even fiat currency) for cryptocurrency. In Section 4.1 the authors describe a technique by which a merchant can choose k so as to transfer this risk to their counterparty.

Note: I could not find a specific publication date for this paper, the 21^st September 2021 date is from the Wayback Machine.

Obadia et al (7^th December 2021)

Unity is strength: A formalization of cross-domain maximal extractable value by Alexandre Obadia, Alejo Salles, Lakshman Sankar, Tarun Chitra, Vaibhav Chellani, and Philip Daian examines the potential for MEV across multiple interconnected blockchains:

In this work, we call each of these interconnected blockchains ‘domains’, and study the manifestation of Maximal Extractable Value (MEV, a generalization of “Miner Extractable Value”) across them. In other words, we investigate whether there exists extractable value that depends on the ordering of transactions in two or more domains jointly.

There are many types of domain:

Layer 1s, Layer 2s, side-chains, shards, centralized exchanges are all examples of domains.

The key to Cross-Domain Maximal Extractable Value is:

MEV extraction has historically been thought of as a self-contained process on a single domain, with a single actor (traditionally the miner) earning an atomic profit that serves as an implicit transaction fee. In a multi-chain future, extracting the maximum possible value from multiple domains will likely require collaboration, or collusion, of each domain’s sequencers if action across multiple domains is required to maximize profit.
...
We expect that, given the deployment of AMMs and other MEV-laden technologies across multiple domains, the benefit of extracting MEV across multiple domains will often outweigh the cost of collusion

Obadia Fig 1

And thus:

Since it exists across domains and given it is finite, the competition for such opportunities will be fierce and it is likely no bridge will be fast enough to execute a complete arbitrage transaction as exemplified in Figure 1.

One observation is that a player that already has assets across both domains does not need to bridge funds to capture this MEV profit, reducing the time, complexity, and trust required in the transaction. This means that cross-domain opportunities may be seized in two simultaneous transactions, with inventory management across many domains being internal to a player’s strategy to optimize their MEV rewards.

And this, for advocates of decentralization, is a problem because:

Such behavior is similar to the practice of inventory management that market makers and bridges in traditional finance do, which primarily consists of keeping assets scattered across multiple heterogeneous-trust domains (typically centralized exchanges), managing risks associated with these domains, and determining relative pricing. Some key differences include the ability to coordinate with actors in the system other than themselves, such as through a DAO or a system like Flashbots. However, given these conclusions, it is likely that traditional financial actors may have a knowledge-based advantage in cross-domain MEV risk management, which may induce centralization vectors that come from such actors being able to run more profitable validators.

Despite being grim, this fact is important as it reveals a key property that cross-domain interactions are subject to: the loss of composability. There is no more atomic execution. This introduces additional execution risk, as well as requires higher capital requirements, further raising the barriers-to-entry required to extract MEV.

This is not the only "negative externality" the authors identify. Another comes from the dominance of large, coin-operated miners or validators, who can afford to dominate in multiple domains at once:

Cross-domain extractable value may create an incentive for sequencers (i.e. validators in most domains) to amass votes across the networks with the most extractable value.

This is especially relevant when realizing there already exist large validators and staking providers running infrastructure across many networks. It is unlikely that those who are for-profit entities will forego access to MEV revenue long term if such revenue is substantial.

Another is the risk of concensus instability:

Time-bandit attacks were first introduced in [Daian et al], and consist in looking at cases where the miner has a direct financial incentive to re-org the chain it is mining on.

In a cross-domain setting, there may now exist incentives to re-org multiple domains, or to re-org weaker domains in order to execute attacks resembling double-spends. This will be particularly relevant to the security of bridges.

Yet another arises from my favorite issue, Economies of Scale in Peer-to-Peer Networks:

One general worry is of the potential economies of scale, or moats, that a trader could create across domains which would end up increasing the barrier to entry for new entrants and enshrine existing players’ dominance.

example. Suppose a domain orders transactions on a first-in first-out basis (FIFO). Such a domain effectively creates a latency race between traders going after the same opportunity. As seen in traditional markets, traders will likely invest in latency infrastructure in order to stay competitive, possibly reducing the efficiency of the market.

If several domains also have such ordering rules, traders could engage in a latency race in each of them, or a latency race to propagate arbitrage across domains, making the geographical points in which these systems operate targets for latency arbitrage. It is likely the infrastructure developed to optimize one domain, can be used across multiple domains. This could create a ‘super’ latency player, and certainly advantages entities which already have considerable expertise in building such systems in traditional finance. Such latency-sensitive systems may erode the security of systems that do not rely on latency, by advantaging latency-optimizing players in the cross-domain-MEV game. This area warrants substantial further study, as it is well-known that global network delays require relatively long block times in Nakamoto-style protocols to achieve security under asynchrony [Pass et al], and it is possible cross-domain-MEV may erode the fairness of validator rewards in such protocols.

And finally, a consequence of Cross-Domain Maximal Extractable Value is:

the introduction of heterogeneous pricing models across different players in the blockchain ecosystem. Previously, MEV was an unambiguous quantity denominated in a common base asset, ETH. In a multi-chain future, the relative price differences between actors is not only relevant in calculating MEV, it can in fact create MEV. For example, a previous validator may leave a system in what is in their pricing model a 0-MEV state, but the next validator, who disagrees with this pricing model, may see opportunities to rebalance MEV to increase assets it subjectively values more. This provides yet another intution for why MEV is fundamental to global, permissionless systems even if they do not suffer from ordering manipulation of the kind described in [Daian et al].

To sum up, the availability of MEV derived from executing transactions in multiple domains greatly advantages large players, especially those with expertise from traditional finance, and increases the risk of consensus instability.

Qin et al (10^th December 2021)

Quantifying Blockchain Extractable Value: How dark is the forest? by Kaihua Qin, Liyi Zhou and Arthur Gervais claims three main contributions:

• We are the first to comprehensively measure the breadth of BEV from known trading activities (i.e., sandwich attacks, liquidations, and arbitrages). Although related works have studied sandwich attacks in isolation, there is a lack of quantitative data from real-world exploitation to objectively assess their severity.
• We are the first to propose and empirically evaluate a transaction replay algorithm, which could have resulted in 35.37M USD of BEV. Our algorithm extends the total captured BEV by 35.18M USD, while intersecting with only 1.43% of the liquidation and 0.11% of the arbitrage transactions
• We are the first to formalize the BEV relay concept as an extension of the P2P transaction fee auction model. Contrary to the suggestions of the practitioner community, we find that a BEV relayer does not substantially reduce the P2P network overhead from competitive trading.

The authors examined trading over 32 months from the 1st of December, 2018 to the 5th of August, 2021 looking for the three types of attacks. As regards sandwich attacks they found:

2,419 Ethereum user addresses and 1,069 smart contracts performing 750,529 sandwich attacks on Uniswap V1/V2/V3, Sushiswap, and Bancor, with a total profit of 174.34M USD (cf. Fig. 3). Our heuristics do not find sandwich attacks on Curve, Swerve, and 1inch. Curve/Swerve are specialized in correlated, i.e., pegged- coins with minimal slippage. Despite the small market cap. (< 1% of Bitcoin), SHIB is the most sandwich attack-prone ERC20 token with an adversarial profit of 6.84M USD.

We notice that 240,053 sandwich attacks (31.98%) are privately relayed to miners (i.e., zero gas price), accumulating a profit of 81.04M USD. Sandwich attackers therefore actively leverage BEV relay systems (cf. Section VI-A) to extract value. We also observe that 17.57% of the attacks use different We also observe that 17.57% of the attacks use different accounts to issue the front- and back-running transactions.
...
Zhou et al. [2] estimate that under the optimal setting, the adversary can attack 7,793 Uniswap V1 transactions, and realize 98.15 ETH of revenue from block 8M to 9M. Based on our data, we estimate that only 63.30% (62.13 ETH) of the available extractable value was extracted.

Qin et al Fig 5

Note that the 32% of non-public sandwich attacks generated 46% of the MEV, so there is an advantage in keeping them private. Clearly, with only 63% of the potential MEV realized, there was room to optimize sandwich trading strategies. As regards liquidations, they examined:

all liquidation events on Aave (Version 1 and 2), Compound, and dYdX from their inception until block 12965000 (5th of August, 2021). We observe a total of 31,057 liquidations, yielding a collective profit of 89.18M USD over 28 months (cf. Fig. 5a and 5b). Note that we use the prices provided by the price oracles of the liquidation platforms to convert the profits to USD at the moment of the liquidation.
...
We identify 1,956 transactions (6.3%) with zero gas price out of the 31,057 liquidation events, implying that liquidators relay liquidation transactions to miners privately without using the P2P network. These privately relayed transactions yield a total profit of 10.69M USD.

The 6.3% of private transactions generated 12% of the MEV. As regards arbitrage, they write:

From the 1st of December, 2018 to the 5th of August, 2021, we identify 6,753 user addresses and 2,016 smart contracts performing 1,151,448 arbitrage trades on Uniswap V1/V2/V3, Sushiswap, Curve, Swerve, 1inch, and Bancor, amounting to a total profit of 277.02M USD. We find that 110,026 arbitrage transactions (9.6%) are privately relayed to miners, representing 82.75M USD of extracted value.
...
ETH, USDC, USDT, and DAI are involved in 99.91% of the detected arbitrages.

Note that the 9.6% of non-public transactions represent 30% of the MEV, so the advantage of keeping arbitrage transactions private is obvious. They also investigated attacks that congest the blockchain:

A clogging attack is, therefore, a malicious attempt to consume block space to prevent the timely inclusion of other transactions. To perform a clogging attack, the adversary needs to find an opportunity (e.g., a liquidation, gambling, etc.) which does not immediately allow to extract monetary value. The adversary then broadcasts transactions with high fees and computational usage to congest the pending transaction queue. Clogging attacks on Ethereum can be successful because 79% of the miners order transactions according to the gas price
...
We identify 333 clogging periods from block 6803256 to 12965000 , where 10 user addresses and 75 smart contracts are involved ... While the longest clogging period lasts for 5 minutes (24 blocks), most of the clogging periods (83.18%) account for less than 2 minutes (10 blocks).

Much longer clogging attacks have happened:

This is what appears to have happened with the infamous Fomo3D game, where an adversary realized a profit of 10,469 ETH by conducting a clogging attack over 66 consecutive blocks.

Blockchain scalability and the fragmentation of crypto by Frederic Boissay et al shows that my Fixed Supply, Variable Demand was correct in suggesting that congestion was necessary to the stability of blockchains. Boissay et al write:

To maintain a system of decentralised consensus on a blockchain, self-interested validators need to be rewarded for recording transactions. Achieving sufficiently high rewards requires the maximum number of transactions per block to be limited. As transactions near this limit, congestion increases the cost of transactions exponentially. While congestion and the associated high fees are needed to incentivise validators, users are induced to seek out alternative chains. This leads to a system of parallel blockchains that cannot harness network effects, raising concerns about the governance and safety of the entire system.

Thus these systems are necessarily vulnerable to clogging attacks.

Capponi et al (11^th February 2022)

In The Evolution of Blockchain: from Lit to Dark, Agostino Capponi, Ruizhe Jia and Ye Wang describe a model built to:

study the economic incentives behind the adoption of blockchain dark venues, where users’ transactions are observable only by miners on these venues. We show that miners may not fully adopt dark venues to preserve rents extracted from arbitrageurs, hence creating execution risk for users. The dark venue neither eliminates frontrunning risk nor reduces transaction costs. It strictly increases payoff of miners, weakly increases payoff of users, and weakly reduces arbitrageurs’ profits. We provide empirical support for our main implications, and show that they are economically significant. A 1% increase in the probability of being frontrun raises users’ adoption rate of the dark venue by 0.6%. Arbitrageurs’ cost-to-revenue ratio increases by a third with a dark venue.

In other words, dark pools transfer MEV mostly from external arbitrageurs to miners, with a smaller transfer to users. But the equilibrium involves continued use of the public mempool because miners enjoy the extra fees arbitrageurs use to attempt to position their trades relative to their victim's trades. Capponi et al describe a pair of tradeoffs. First, for blockchain users:

On the one hand, using the dark venue alone presents execution risk to users. Transaction submitted to the dark venue face the risk of not being observed by the miner updating the blockchain, who may not have adopted the dark venue. On the other hand, users who only submit through the dark venue avoid the risk of being frontrun.

Second, for arbitrageurs:

Arbitrageurs who only use the dark venue would not leak out information about the identified opportunity to their competitors. They also gain prioritized execution for their orders because miners on the dark venue prioritize transactions sent through such venue. ... If instead the execution risk is high, arbitrageurs will use both the lit and the dark venue: through the dark venue they gain prioritized execution, and through the lit venue they are guaranteed execution.

Their model predicts that:

both arbitrageurs and the frontrunnable user will submit their transactions through the dark venue, if sufficiently many miners adopt it.

There is a weakness in their model hiding behind "sufficiently many miners". Their model of mining consists of:

a continuum of homogeneous, rational miners. All miners have the same probability of earning the right to append a new block to the blockchain.

But this does not model the real world. Miners are not the ones who chose transactions from the lit or dark pools, the choice is made by mining pools, and a small number of large pools have a much higher "probability of earning the right to append a new block to the blockchain". Thus the use of the dark venue depends upon whether the large mining pools adopt it. Given that typically 2 or 3 pools dominate Ethereum mining, if they adopt the dark pools usage will be high and users of the lit venue will be disadvantaged.

The authors also study dark pool usage empirically:

Our dataset contain dark venue transaction-level data of Ethereum blockchain collected from Flashbots API, Ethereum block data, and transaction-level data from Uniswap V2 and Sushiswap AMMs. Our empirical analysis confirms that the dark venue is partially adopted, and further estimates the dark venue adoption rate around 60% as of Jul 2021. Our analysis also shows that miners who join the dark venue have higher revenue than those who stay on the lit venue.

The methodology described in Seciton 6.3 suggests that what the authors mean is that about 60% of the hash rate has adopted the dark pools; for Ethereum this would be no more than 4 mining pools.

Elder (10^th June 2022)

Bryce Elder's How should we police the trader bots? is about bots trading in the conventional markets, but the message should be heeded by the developers of cryptocurrency trading bots:

An overarching rule of securities legislation is that market abuse is market abuse, irrespective of whether it’s committed by a human or machine. What matters is behaviour. An individual or firm can expect trouble if they threaten to undermine market integrity, destabilise an order book, send a misleading signal, or commit myriad other loosely defined infractions. The mechanism is largely irrelevant.

And importantly, an algorithm that misbehaves when pitted against another firm’s manipulative or borked trading strategy is also committing market abuse. Acting dumb under pressure is no more of an alibi for a robot than it is for a human.

For that reason, trading bots need to be tested before deployment. Firms must ensure not only that they will work in all weathers, but also that they won’t be bilked by fat finger errors or popular attack strategies such as momentum ignition. The intention here is to protect against cascading failures such as the “hot potato” effect that contributed to the 2010 flash crash, where algos didn’t recognise a liquidity shortage because they were trading rapidly between themselves.

Mifid II (in force from 2018) applies a very broad Voight-Kampff test. Investment companies using European venues are obliged to ensure that any algorithm won’t contribute to disorder and will keep working effectively “in stressed market conditions”. The burden of policing falls partly on exchanges, which should be asking members to certify before every deployment or upgrade that bots are fully tested in “real market conditions”.

It is easy to mandate testing but not so easy to do it. Running the bot against market history is easy but doesn't actually prove anything because the history of trades doesn't reflect the effect on prices of the bot's trades, and thus doesn't reflect the response of all the other bots to this bots trades, and their effects on prices, which would have caused both this bot and all the other bots to respond, which ...

And, even if realistic testing were possible, since the DEX are "decentralized" they have no way to require that bots using them are tested, or how they are tested. So, based on the history of conventional markets, even if cryptocurrencies were not infested with pump-and-dump schemes, flash crashes and the resulting market disruptions are inevitable.

Auer et al (16^th June 2022)

Auer et al Graph 1

Raphael Auer, Jon Frost and Jose Maria Vidal Pastor's Miners as intermediaries: extractable value and market manipulation in crypto and DeFi provides a readable overview of MEV:

Far from being “trustless”, cryptocurrencies and decentralised finance (DeFi) rely on intermediaries who must be incentivised to maintain the ledger of transactions. Yet each of the validators or “miners” updating the blockchain can determine which transactions are executed and when, thus affecting market prices and opening the door to front-running and other forms of market manipulation.

Auer et al explain:

MEV can hence resemble illegal front-running by brokers in traditional markets: if a miner observes a large pending transaction in the mempool that will substantially move market prices, it can add a corresponding buy or sell transaction just before this large transaction, thereby profiting from the price change (at the expense of other market participants). Miners can also engage in “back-running” or placing a transaction in a block directly after a user transaction or market-moving event. This could entail buying new tokens just after they are listed, eg in automated strategies from multiple addresses, to manipulate prices. Finally, miners can engage in “sandwich trading”, where they execute trades both before and after a user, thus making profits without having to take on any longer-term position in the underlying assets.

Auer et al Graph 2

They estimate the scale of the problem:

Since 2020, total MEV has amounted to an estimated USD 550–650million on just the Ethereum network, according to two recent estimates (Graph 2, left-hand panel). In addition to sandwich attacks, MEV results from liquidation attacks (ie forcing liquidations), replay attacks (cloning and front-running a victim’s trade) and decentralised exchange arbitrage (right-hand panel; online appendix). Notably, these estimates are based on just the largest protocols and are hence likely to be understated. Thus, the amount of MEV captured in the data is only one portion of the total profits that miners can extract from other users.

Source

Extracting data from this chart and assuming that the $550-650K represents "price" at the time, it appears that over the same period Ethereum miners earned about $27,616K, so MEV represented only about 2.4% additional mining income. This sounds small, but most of the time Proof-of-Work mining is a low-margin business so 2.4% extra income could well be significant. It isn't clear how much capital is needed to exploit MEV, so 2.4% needs to be decreased by the cost of the required capital.

Of course, this being Ethereum, capturing MEV has been automated:

"Bots” that exploit MEV are now active on different decentralised exchanges. This imposes a fixed cost of mining, encouraging concentration. Solutions such as moving to dark venues, where transactions are only visible to miners, have not so far reduced front-running risk (Capponi et al (2022)). The additional fees and unpredictability for users mean an additional form of insider rents in DeFi markets.

And things are only going to get worse:

Looking forward, MEV could intensify. Indeed, in general equilibrium, miners may be forced to engage in MEV to survive. Miners who engage in MEV will on average make higher profits and buy more computing power, and they could thus eventually crowd out miners who do not. Thus, a form of rat race develops from the combination of the competitive and decentralised nature of updating and the fact that every miner can assemble their block any way they want. It has been argued that MEV forms an existential risk to the integrity of the Ethereum ledger (Daian et al (2019); Obadia (2020)).