Engineering a bitcoin max_block_size

Engineering #

The crucial and unique task of the engineer is to identify, understand, and interpret the constraints on a design in order to produce a successful result.

Constraints #

An engineer finds an acceptable solution within the given constraints of the problem.

For most problems solved by engineering, an upper limit on what may be achieved is set by budget, schedule and technological limitations; the lower limit is determined by the specific functional requirements of the project.

To engineer a useful bitcoin MAX_BLOCK_SIZE value, the constraints must be defined, or there can be no progress made.

Should max block size change? #

Before addressing the constraints, an important question must be answered: should there even be any work undertaken?

The answer is maybe. The current value was chosen without any apparent engineering and may or may not be appropriate. Determining an appropriate max block size is the goal of this post.

Upper limit #

What factors stop us setting MAX_BLOCK_SIZE = 340 GB?

Block propagation time #

If a block takes more than ten minutes to fetch, there is no way to fetch all the blocks since they will be generated faster than they can be received.

The average global connection speed is 5.6 Mb/s

In ten minutes, an average internet user can download 3360 Mb, or 420 MB. This would be a total of 1874 GB per month, which may impinge on bandwidth caps depending on the user.

The bandwidth must also be shared with other functions of the bitcoin network, such as transaction propagation, which will lower the available bandwidth for block propagation.

Assumptions:

• Full nodes are run by average users, not specialist users with higher bandwidth, nor novice users with lower bandwidth.
• Users running full nodes make 100% of their bandwidth available for bitcoin usage.
• Upload and download bandwidth are equally available and demanded

Blocks cannot be larger than 420 MB given current network technology.

Block verification time #

If a block takes longer than ten minutes to verify, there is no way to verify all the blocks since they will be generated faster than they can be verified.

Altoidnerd recently published their log for verifying the blockchain. It took 14644s to process up to block height 394395, which was created on the 22nd January 2016. This processed about 105 million transactions, which makes for a processing rate of about 7000 transactions per second, or 4.2 million in ten minutes.

The total blockchain size at height 394395 was around 55 GB. This means an average of 524 bytes per transaction.

4.2 million transactions per block would require 2100 MB blocks. Regardless of the specific computer used for processing, cpu is not going to be the bottleneck on the upper bound of block size, since there’d be bandwidth issues well before then.

These calculations are in the same order of magnitude with other simulations which estimate 1.1 seconds to verify 8 MB of transactions, which leads to a limit of 4364 MB per block.

Blocks cannot be larger than 2100 MB given existing consumer computation of transaction signatures.

Storage limits #

If blocks are larger than can be stored on a hard drive, storing the blockchain would be impossible.

Determining an upper limit on block size based on hard drive space depends on budget constraints, which depend on the user running the full node.

The current cost of storage is less than 3c per gigabyte. At the current rate of 1 MB every ten minutes, the storage cost of taking part in bitcoin is $1.58 per year.

There is no way storage is ever going to be the bottleneck for the upper limit on block size. It does entail some ongoing cost, but it’s far from prohibitive.

Decentralization objectives #

The larger the blockchain, the less full nodes are likely to be run. This causes centralization and goes directly against the key benefit of bitcoin over existing solutions.

The main objective stated for bitcoin comes from the whitepaper: “allow online payments to be sent directly from one party to another without going through a financial institution”. Can we use this metric in determining the upper limit of the block size? When does the block size become so large that it breaks this objective?

It depends on the definitions

• directly - is lightning network direct?
• financial institution - does a financial institution running a full node mean bitcoin has failed to meet the goals?

Lower limit #

What factors stop us setting MAX_BLOCK_SIZE = 1 kB?

Transaction rate and transaction backlog #

A low block size decreases the transaction rate, which lowers the utility of bitcoin. It amplifies the effect of transaction prioritization schemes which are not enforceable by network consensus (ie only enforceable by miner rules, which can be arbitrary).

Backlogged transactions (ie not included in the blockchain, only in the mempool) pose a risk to receivers and an inconvenience to spenders. Large backlogs lead to increased complexity for all bitcoin users, such as the replace-by-fee feature.

Demand for space on the bitcoin network has consumed about 35,000 MB in the year between May 2015 to May 2016.

This sets a lower bound of about 660 kB per block to meet the current demand.

There are lots of factors in how low the block size can go whilst still satisfying transaction demand:

• use trustless off-chain solutions such as the lightning network, which have stated 133 MB blocks are needed to scale to 7 billion users (slide 51 of the lightning network).
• use trusted (centralized) off-chain services to keep bitcoin decentralized (eg coinbase-to-coinbase transactions), but at the disadvantage of no longer using a fully trustless system.

Bitcoin would benefit from a clear set of goals and metrics for the consumer use of centralized vs decentralized and on-chain vs off-chain solutions.

To provide a welcome-to-all solution utilizing trustless off-chain transactions requires a lower bound of 133 MB on max block size.

Mining fees #

Transaction fees ensure mining remains profitable and the network remains sustainable into the future. The total amount of transaction fees has implications for the viability of mining and the security of bitcoin transactions.

The heuristic for the relation of max block size with transaction fees is low block size = few txs * high fees vs high block size = many txs * low fees. In theory they should balance out, meaning block size is irrelevant for overall miner fees.

It still leaves the question unanswered: which max block size is best to encourage participation in mining, and thus ensure the future sustainability and security of bitcoin?

• Consumer behavior seeks low fees; this implies they desire spacious blocks, ie larger blocks
• Miner behavior seeks high fees; this implies they desire full blocks, ie just smaller than the demand requires

Is there an optimum value for max block size which causes the relation between miners and consumers to remain stable and optimum? Since this relation is currently poorly understood, the appropriate scientific or engineering response is to observe past behaviors, combine it with desired outcomes, and iteratively adjust. There is no exact answer in this case.

A lower limit of max block size based on mining fees cannot be established without clear goals for mining and consumer fees and behavior. The bitcoin project would benefit from a clear policy around expectations for consumer and mining fees to ensure sustainability into the future, as well as open monitoring of progress toward the goals and subsequent adjustments.

A note about the lower bound #

The lower bound is determined by minimum functionality rather than technical limitations. Since the minimum functionality depends only on what individuals believe to be the required function, it’s less clear how to establish a lower limit than it is to establish an upper limit.

The upper limit has hard technological parameters determining how far it can go. There can be improvements with better software etc, but there are clear limits.

The uncertainty of ‘how low’ is driven by belief of what bitcoin should be. But that has never been clearly decided or defined. There are no clear goals nor clear metrics as to whether those goals are being achieved.

I don’t believe discussing the value of the lower limit is a particularly useful discussion. Instead, bitcoin should work toward creating clear goals and metrics that indicate when the lower bound has become inadequate; this would definitely be useful.

The complex governance model of bitcoin becomes irrelevant if data from engineering can demonstrate a difference between the goals vs observed reality of bitcoin. This can only be done when clear goals are in place.

Caveats #

Bitcoin is a complex system with many considerations.

Engineers design for every-day use cases (as discussed in this post), but also for the edge cases (usually adding a risk factor on top). Designing for failure is an essential part of engineering.

There are some important considerations for max block size:

• Some transactions take much longer to verify, as seen in theory and in reality. This has implications for the upper limit due to transaction verification.
• Unspent Transaction Output growth has an impact on the performance of the network, and is affected by max block size.
• The risk of various failure modes must be quantified. Most failures do not result in complete destruction of bitcoin, and changes should be engineered with appropriate attention to the risk. The consequence of risk has been seen in previous hard forks, and despite being serious, it does not result in complete destruction of the system.
• The reversibility of decisions must be accounted for. Bitcoin is capable of responding to feedback. There are some excellent metrics for the performance of bitcoin, and excellent systems for responding to that feedback. Bitcoin must implement appropriate responses to feedback, ensure those responses generate meaningful signals in the future, and respond to signals in a timely manner; doing so will make bitcoin stronger. It seems the feedback signal for ‘transaction demand’ is strong, but the response is lacking. Claiming ‘no action required’ is simply false.

Existing analysis and solutions #

There are many existing proposals with calculations for setting an appropriate block size, and these contain a wealth of information around the history and current state-of-the-art for this controversial topic.

• BIP-101 - 8 MB doubling every two years up to 8 GB
• BIP-102 - 2 MB one-time increase
• BIP-103 - Dynamic by time, up to 17.7% change per year
• BIP-105 - Dynmaic by vote, between 1 MB and 8 MB
• BIP-106 - Dynamic by consumption rate
• BIP-107 - Up to 6 MB to 2020, then dynamic, up to 10% per month

Conclusion #

133 MB < max block size < 420 MB

The lower bound is set by what users believe to be the goals of bitcoin. In this case the belief used is ‘welcome-to-all with infrequent on-chain transactions and frequent off-chain transactions’.

The upper bound is set by current technological limitations.

The possible bounds are:

The goal of this document is to put some rigor around the possible choice of a value for MAX_BLOCK_SIZE, and apply an engineered solution to fill the void created by ‘picking a magic number’ for max block size.

However the big takeaway from this is the need to determine clear goals and metrics for the bitcoin project, so there can be feedback on whether implemented changes are achieving the desired effect. Without this, the project cannot establish suitable constraint conditions and cannot progress beyond the current state; this creates a political situation where there should be an engineering solution.