BIP 54: Consensus Cleanup

  BIP: 54
  Layer: Consensus (soft fork)
  Title: Consensus Cleanup
  Author: Antoine Poinsot <mail@antoinep.com>
          Matt Corallo <bips@bluematt.me>
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0054
  Status: Draft
  Type: Standards Track
  Created: 2024-04-11
  License: CC0-1.0

Abstract

This document proposes new consensus rules in order to fix the timewarp attack, reduce the worst case block validation time, prevent Merkle tree weaknesses, and avoid duplicate transactions without bip-0030 validation.

Motivation

This proposal addresses a number of long-standing vulnerabilities and weaknesses in the Bitcoin protocol. Bundling these fixes together allows to overcome the fixed cost of deploying a Bitcoin soft fork.

The timewarp bug permits a majority hashrate attacker to arbitrarily increase the block rate, allowing them to steal block subsidy from future miners and increase validation costs to nodes that have to deal with the increased average transaction rate. By strategically setting the block timestamp, the timewarp bug lets miners bring down the difficulty to its minimum within 38 days of starting the attack. The existence of this bug not only significantly empowers a 51% attacker, but also makes it notably harder to reason about miners' incentives. It could indeed be in the interest of short-sighted miners as well as short-sighted users to exploit this vulnerability in a small enough proportion to increase the block rate without fatally hurting the network, as the effectively increased block space would — all other things being equal — bring fee rates down for users.

Specially crafted blocks may be expensive to process, with validation times ranging from a few minutes up to more than an hour on lower-end devices. Long block validation times are a nuisance to users, increasing the cost to independently fully validate the consensus rules. In addition they can be used by miners to attack their competition, creating perverse incentives, centralization pressures and leading to reduced network security.

In computing a block's Merkle root, a 64-byte transaction can be interpreted as an intermediate node in the tree in addition to a leaf. This makes it possible to fake inclusion proofs by pretending a 64-byte block transaction is an inner node, as well as to pretend the inner nodes on one level of the tree are the actual block transactions.

Since bip-0034 activation, explicit bip-0030 validation is not necessary until block height 1,983,702¹. Mandating new coinbase transactions be different from the early bip-0034 violations makes it possible to get rid of bip-0030 validation forever. Besides its unnecessary cost, another downside of bip-0030 validation is that it cannot be performed by Utreexo clients. Finally, leveraging the coinbase transaction's nLockTime field allows applications to recover the block height corresponding to a coinbase transaction without having to parse Script.

Specification

For all blocks after activation the following new rules apply.

Given a block at height N:

if N % 2016 is equal to 0, the timestamp of the block must be set to a value higher than or equal to the value of the timestamp of block at height N-1 minus 7200 (T_N ≥ T_N−1 − 7200);
if N % 2016 is equal to 2015, the timestamp of the block must be set to a value higher than or equal to the value of the timestamp of the block at height N-2015 (T_N ≥ T_N−2015).

A limit is set on the number of potentially executed signature operations in validating a transaction. It applies to all transactions in the block except the coinbase transaction². For each input in the transaction, count the number of CHECKSIG and CHECKMULTISIG in the input scriptSig and previous output's scriptPubKey, including the P2SH redeemScript. The accounting is the same as for bip-0016: a CHECKSIG/CHECKSIGVERIFY operation accounts for 1 and a CHECKMULTISIG/CHECKMULTISIGVERIFY accounts for the number of public keys associated, or 20 if the number of public keys is greater than 16. A CHECKMULTISIG/CHECKMULTISIGVERIFY not directly preceded by a minimally-pushed number between 1 and 16 (included) accounts for 20. If the total is strictly higher than 2500, the transaction is invalid.

Transactions whose witness-stripped serialized size is exactly 64 bytes are invalid.

The coinbase transaction's nLockTime field must be set to the height of the block minus 1³ and its nSequence field must not be equal to 0xffffffff.

Rationale

The restrictions on the timestamp of the first and last blocks of a difficulty adjustment period fix the timewarp and Murch–Zawy vulnerabilities⁴. The latter poses mostly theoretical concerns but is extremely low risk to fix: the duration of an adjustment period has never been, and should never be, negative. The former is fixed by preventing the timestamp of the first block of a difficulty period from being lower than the previous block's, with a two-hour grace period. A previous proposal to fix timewarp used a ten-minute grace period instead, also adopted for testnet4. Out of an abundance of caution and because it only trivially worsens the block rate increase under attack, a two-hour grace period is used here⁵.

Disabling some Script operations and functionalities was previously proposed to reduce the worst case block validation time but was met with resistance due to confiscation concerns⁶. A delicate balance needs to be struck between minimizing the confiscation risks of a mitigation, even if merely theoretical, and bounding the costs one could impose on all other users of the system. To this effect a limit on the number of potentially executed signature operations pinpoints exactly the harmful behaviour, leaving maximum flexibility in how Script functionalities may have been used. Such a limit reduces the worst case block validation time by a factor of 40 and drastically increases the preparation cost of an attack to make it uneconomical for a miner⁷. The maximum of 2500 was chosen as the tightest value that did not make any non-pathological standard transaction invalid⁸.

In the presence of 64-byte transactions a block header's Merkle root may be valid for different sets of transactions. This is because in the Merkle tree construction a 64-byte transaction may be interpreted as the catenation of two 32-byte hashes, or the catenation of two 32-byte hashes may be interpreted as a transaction. The former allows to fake a block inclusion proof and the latter makes it such that for a valid block the Merkle root in the block header is not a unique identifier for the corresponding list of valid transactions⁹. 64-byte transactions can only contain a scriptPubKey that lets anyone spend the funds, or one that burns them. 64-byte transactions have also been non-standard since 2019. It was suggested that the known vulnerabilities could instead be mitigated by committing to the Merkle tree depth in the header's version field¹⁰. The authors believe it is preferable to address the root cause by invalidating 64-byte transactions. This approach also fixes the vulnerability without developers of SPV verifiers having to implement the mitigation or to know it is necessary in the first place.

Several blocks prior to bip-0034 activation contain a coinbase transaction whose scriptSig contains a valid bip-0034 commitment to a future block height. This offers an opportunity to duplicate these coinbase transactions in the future¹¹ and for this reason bip-0030 validation will need to be re-activated from block 1,983,702. A simple way to prevent this is to mandate that future coinbase transactions vary from coinbase transactions before bip-0034 activation. There are multiple ways of achieving this, but setting and enforcing the timelock for the coinbase transaction makes it so all coinbase transactions past Consensus Cleanup activation could not have been valid before this height and therefore cannot be a duplicate¹².

Backward compatibility

This proposal only tightens the block validation rules: there is no block that is valid under the rules proposed in this BIP but not under the existing Bitcoin consensus rules. As a consequence these changes are backward-compatible with non-upgraded node software. That said, the authors strongly encourage node operators to upgrade in order to fully validate all consensus rules.

Miner forward compatibility

Bitcoin Core version 29.0 and later will not generate a block template that violates the timestamp restrictions introduced in this BIP. Although it would be extremely unlikely due to the grace period used in this proposal, miners should use the curtime or mintime field from the getblocktemplate result for their block's timestamp to make sure they always create blocks valid according to this proposal. Note this is not a new requirement: using a timestamp lower than the mintime field from the getblocktemplate result already leads to creating an invalid block.

Bitcoin Core as of version 29.0 may relay and create a block template including a transaction that violates the signature operations limit introduced in this BIP. A newer version of Bitcoin Core that makes this type of transaction non-standard should be widely adopted before this soft fork is considered for activation.

Bitcoin Core version 0.16.1 and later will neither relay nor create block templates that include 64-byte transactions.

The coinbase transaction is usually crafted by mining pool software. To the best of the authors' knowledge, there does not exist an open source reference broadly in use today for such software. We encourage mining pools to update their software to craft coinbase transactions that are forward-compatible with the changes proposed in this BIP.

Acknowledgements

This document builds upon an earlier proposal by Matt Corallo.

The authors would like to thank everyone involved in researching the most appropriate mitigation for each of these bugs. We would like to thank in particular Anthony Towns and Sjors Provoost for their direct contributions to this proposal, as well as @0xb10c and Brian Groll for providing the authors with data to analyze the proposed mitigations.

Copyright

This document is licensed under the Creative Commons CC0 1.0 Universal license.

Block 1,983,702 is the earliest future block which could contain a duplicate coinbase transaction while still respecting bip-0034. See this post for a list of all such future blocks. ²: Technically this limit cannot apply to a coinbase transaction as the size of its sole input's scriptSig is limited. ³: The locktime validation, which is also performed for coinbase transactions, enforces that the nLockTime value is the last block at which a transaction is invalid, not the first one at which it is valid. ⁴: The timewarp attack is described here and the Murch–Zawy attack here. ⁵: The testnet4 difficulty exception pushed blocks' timestamps in the future when abused, revealing how some broken pool software may produce blocks that don't respect a 10 minutes grace period. Some raised concerns similarly broken software might be used on mainnet. Using a grace period of 2 hours instead of 10 minutes only reduces the expected block interval time under attack by ~2.2 seconds. See this post for more. ⁶: The argument is about someone having a timelocked presigned transaction using some of those features in its output script. The transaction cannot be mined before activation. Such outputs would not be covered by an amnesty for old UTxOs. See for instance here and here for discussions on this topic. ⁷: It is important to reduce the worst case block validation time as well as the ratio of validation time imposed over preparation cost. The former is to limit the damages an externally motivated attacker can do. The latter is to disincentivize miners slowing down their competition by mining expensive blocks. See this thread for more. ⁸: A non-pathological transaction would have a public key per signature operation and at least one signature per input. Per standardness a single P2SH input may not have more than 15 signature operations. Even by using 1-of-15 CHECKMULTISIGs a transaction would bump against the maximum standard transaction size before running into the newly introduced limit. To run against the newly introduced limit but not the transaction size a transaction would need to spend P2SH inputs with a redeem script similar to CHECKSIG DROP CHECKSIG DROP .... This type of redeem script serves no purpose beyond increasing its validation cost, which is exactly what this proposal aims to mitigate. ⁹: See this writeup by Suhas Daftuar for an explanation as well as a discussion of the consequences. ¹⁰: By Sergio Demian Lerner in a blog post surfaced by Eric Voskuil. Eric also pushed back against the importance of fixing this issue. See this post for an attempt at summarizing the arguments for both sides of this debate. ¹¹: See here for a full list of the heights of historical blocks including a valid bip-0034 height commitment and the corresponding future block height. ¹²: Technically it could be argued a duplicate could in principle always be possible before block 31,001 when nLockTime enforcement was originally soft-forked. But treating coinbase transactions as not having duplicate past Consensus Cleanup activation would be consistent for any implementation which enforces nLockTime from the genesis block, which is the behaviour notably of Bitcoin Core but also of all other implementations the authors are aware of.