No excitement for Bitcoin Classic, Segregated Witness could be a disaster
Posted: Mon Feb 08, 2016 8:06 pm
I had planned to be active last weekend during a four day vacation, but unfortunately I contracted an illness and accomplished little. I began worsening on Wednesday night, and started recovering on Monday, so that the worst of the illness fell on the four days during which I would have been able to work on this pool and on the litecoin BIP101 code. Because four days of productivity were lost, the BIP101 project was set back by at least two weekends. As I'll describe below, the actual release date may stretch into March or even April, given the extreme difficulty I've encountered in testing.
But first, a discussion of Bitcoin Classic is warranted. When I recovered and started following bitcoin again, discussion of Classic had started to increase. The expected initial rush of nodes occurred, but since that rush, node count has leveled off. Perhaps it will again increase if blocks start being mined with it, but there are far fewer nodes supporting Bitcoin Classic now than supported BIP101 when BIP101 was first released. In addition, the increase in Classic nodes does not come at the expense of Core nodes. Instead, the total number of nodes has increased, suggesting that people who support Classic are turning on nodes for the first time - meaning that these new nodes count only half as much as a Core node switching to Classic.
One of the reasons I suspect that Bitcoin Classic will fail is because of the excitement factor. BIP101 is exciting: it permanently resolves a problem and sets bitcoin on a growth trajectory for the indefinite future. Bitcoin Classic is not; it's hard to get people excited over a one-time increase that even its proponents suggest is a large compromise.
Because of that, I offer that most people simply don't care. Like us, they have been following this ridiculous debate for years, and they were excited by solutions like BIP101 and Bitcoin Unlimited. Now, a far less interesting idea is being proposed, and many people have given up and moved on to altcoins or other industries. In addition to block propagation due to lack of nodes being an issue, having only 72% of pools supporting Bitcoin Classic is insufficient for adoption. Many of these pools will offer mining with both forks. As with all software, the defaults will be critical. Since it's unreasonable to expect most of these pools to set the default to Classic, far less than 72% of the network will express support even if every single pool honors its commitment.
Therefore, I continue work on the BIP101 implementation for litecoin and altcoins. Currently, I implemented the changes in Andresen's pull request, got them to compile, got all the unit tests to run successfully, got all the Python integration tests to start, and have progressed to about 3/4 of the "big blocks" tests. Two days ago, I resolved an issue where blocks would not propagate between the test nodes - it turns out that there is a "-mocktime" argument that was implemented in a separate pull request by Gavin Andresen of which I had been previously unaware. Adding the argument fixed that problem, but now I've encountered a new issue where mined blocks are being rejected by the daemons because, of all reasons, the proof of work of the blocks is too easy. I've been at this error for about nine hours, and total work on BIP101 has now surpassed 75 hours.
One of the frustrations with working on code like this is that none of the many errors I've encountered during the testing are related at all to the increased block sizes. 85% of the 75 hours has been spent on configuration and resolving unrelated issues that arise due to trying to understand the incredible complexity of the system. Here are the most difficult challenges I encountered so far:
To this point, I've focused my criticism of Segregated Witness on how external services would need to be rewritten to handle it, but I was ignorant of exactly how much risk there was in simply getting it working in the core codebase. It's already insanely difficult to understand how transactions are serialized into blocks and to be absolutely sure that this security-critical system performs all the proper checks. It is very easy for someone to miss adding a check for an edge case and the entire network would start validating blocks with unintended consequences, all for a likely 10-20% capacity gain. Few people seem to be considering that the bitcoin client is already exceeding what a single human can manage and understand.
In conclusion, while I continually expect to find something that will prove my opinions wrong, during this process I have often been surprised at how I underestimated some of the things I've believed. Bitcoin Classic had a lot of hype, but there is not yet any evidence that a supermajority of miners will support it, and there is also no evidence that any Core nodes are changing to Classic. I thought it would be difficult to configure and compile Litecoin, but I never anticipated how difficult getting the other things in the tests that aren't related to big blocks to pass would be. The BIP101 task is far more difficult to implement than I expected, and the codebase of the daemons is so fragile, that adding complex new features to solve a trivial problem has far more risk than the possible benefit.
The same is true of continual hard forks to increase the blocksize by a little each time. The odds are high enough that at some point, someone is going to make a critical mistake.
But first, a discussion of Bitcoin Classic is warranted. When I recovered and started following bitcoin again, discussion of Classic had started to increase. The expected initial rush of nodes occurred, but since that rush, node count has leveled off. Perhaps it will again increase if blocks start being mined with it, but there are far fewer nodes supporting Bitcoin Classic now than supported BIP101 when BIP101 was first released. In addition, the increase in Classic nodes does not come at the expense of Core nodes. Instead, the total number of nodes has increased, suggesting that people who support Classic are turning on nodes for the first time - meaning that these new nodes count only half as much as a Core node switching to Classic.
One of the reasons I suspect that Bitcoin Classic will fail is because of the excitement factor. BIP101 is exciting: it permanently resolves a problem and sets bitcoin on a growth trajectory for the indefinite future. Bitcoin Classic is not; it's hard to get people excited over a one-time increase that even its proponents suggest is a large compromise.
Because of that, I offer that most people simply don't care. Like us, they have been following this ridiculous debate for years, and they were excited by solutions like BIP101 and Bitcoin Unlimited. Now, a far less interesting idea is being proposed, and many people have given up and moved on to altcoins or other industries. In addition to block propagation due to lack of nodes being an issue, having only 72% of pools supporting Bitcoin Classic is insufficient for adoption. Many of these pools will offer mining with both forks. As with all software, the defaults will be critical. Since it's unreasonable to expect most of these pools to set the default to Classic, far less than 72% of the network will express support even if every single pool honors its commitment.
Therefore, I continue work on the BIP101 implementation for litecoin and altcoins. Currently, I implemented the changes in Andresen's pull request, got them to compile, got all the unit tests to run successfully, got all the Python integration tests to start, and have progressed to about 3/4 of the "big blocks" tests. Two days ago, I resolved an issue where blocks would not propagate between the test nodes - it turns out that there is a "-mocktime" argument that was implemented in a separate pull request by Gavin Andresen of which I had been previously unaware. Adding the argument fixed that problem, but now I've encountered a new issue where mined blocks are being rejected by the daemons because, of all reasons, the proof of work of the blocks is too easy. I've been at this error for about nine hours, and total work on BIP101 has now surpassed 75 hours.
One of the frustrations with working on code like this is that none of the many errors I've encountered during the testing are related at all to the increased block sizes. 85% of the 75 hours has been spent on configuration and resolving unrelated issues that arise due to trying to understand the incredible complexity of the system. Here are the most difficult challenges I encountered so far:
- There are pull requests upon which BIP101 depends that weren't even present in bitcoin at the time of its proposal. They are simple changes, but it took 10 hours just to learn that they existed.
- There are environment variables in the tests that are named "BITCOIND," which I guess had never been changed by the litecoin developers. The tests assign a default value of "bitcoind" if the environment variable is not set, which is really, really bad. Instead, an error should occur stating that the environment variable needs to be defined.
- An error message appears in the debug.log files stating that blocks are unable to be mined because the "hash doesn't match nBits." These errors seem to occur even when mining small blocks, which confuses me as to how these tests even worked in the first place.
To this point, I've focused my criticism of Segregated Witness on how external services would need to be rewritten to handle it, but I was ignorant of exactly how much risk there was in simply getting it working in the core codebase. It's already insanely difficult to understand how transactions are serialized into blocks and to be absolutely sure that this security-critical system performs all the proper checks. It is very easy for someone to miss adding a check for an edge case and the entire network would start validating blocks with unintended consequences, all for a likely 10-20% capacity gain. Few people seem to be considering that the bitcoin client is already exceeding what a single human can manage and understand.
In conclusion, while I continually expect to find something that will prove my opinions wrong, during this process I have often been surprised at how I underestimated some of the things I've believed. Bitcoin Classic had a lot of hype, but there is not yet any evidence that a supermajority of miners will support it, and there is also no evidence that any Core nodes are changing to Classic. I thought it would be difficult to configure and compile Litecoin, but I never anticipated how difficult getting the other things in the tests that aren't related to big blocks to pass would be. The BIP101 task is far more difficult to implement than I expected, and the codebase of the daemons is so fragile, that adding complex new features to solve a trivial problem has far more risk than the possible benefit.
The same is true of continual hard forks to increase the blocksize by a little each time. The odds are high enough that at some point, someone is going to make a critical mistake.