Real world consequences of the Bitcoin blocksize stalemate
Posted: Fri Feb 24, 2017 10:41 am
The Bitcoin block size debate seems to be waged largely in an abstract sense using theoretical arguments about things that could possibly happen in the future. Mining centralization, for example, seems to come up quite often, as do issues like the size of the blockchain and what Satoshi Nakamoto's vision was. Meanwhile, actual consequences are occurring right now, and because few seem to recognize or understand them, I thought that it might be worthwhile to examine the effects the Bitcoin network's dysfunction have been having on actual users.
Not all of these effects are obvious and some will likely surprise people who believed the stalemate would result in different outcomes.
-------------------
1. Changes in payout requests
Transaction fees are bad for business and complex to users, which is why corporations like Coinbase hide them. At Prohashing, we also aim to ensure that users don't have to worry about transaction fees if they don't want to. To accomplish that objective, we pay users for free after their balances rise above a threshold amount in their requested currencies. This "payout threshold" is targeted so that 0.5% of the payout will be lost to fees. The payout threshold used to be a flat $1 for all coins. It remains $1 for all coins except Bitcoin, which was raised to $6, $12, and now stands at $20 (viewtopic.php?f=4&t=1095), because our transactions now cost about $3.10 and there are about 40 outputs.
Each time we raised the threshold, we expected that people would either choose to pay the fee at the prior threshold, stop mining with us, or simply allow us to hold more of their money and get paid less frequently in larger amounts. Instead, what we've found is that the payout thresholds have led to a significant decrease in the usage of Bitcoin. In fact, an examination of the "Payout coins owed" chart at https://prohashing.com demonstrates that customers requesting payment in bitcoins often falls into the minority (today appears to be an abnormality, because one large miner switched his payout coin apparently due to the bubble).
When I asked some larger customers why they changed currencies, the answer was straightforward, but obvious in retrospect. Waiting for larger payouts is not a big deal to them. The problem is that they don't want their money "stuck" on a non-functional network. To see why, consider that for every transaction we sent to a person, he or she has to pay a much greater amount to transfer the money to someone else, because there is a set overhead to transactions. Since money is useless unless it can be spent, the value of each output on the Bitcoin blockchain is reduced by an amount of at least (2 * current transaction fee). That pays to ignore the threshold and then pay fees again to spend the money.
That might be the end of it, except that it's very unlikely that the bitcoins are only penalized by a multiple of the current fees. The output isn't likely to be spent today; it is likely to be spent days, months, or years from now. Let's assume that median fees rise to $20.00 by the end of the year, which I consider to be likely given the current rate of growth. Our customers recognize that their $20.00 payouts today will have no value whatsoever by the end of the year, so they are choosing not to be paid in bitcoins.
This is why fewer than 50% of people now request bitcoins as payments. Altcoins that used to be considered minor players, like Gamecredits and Gulden, are quickly picking up the slack, and Litecoins, which have proven to be a much more stable store of value than bitcoins over the past few months, are gaining traction as well.
2. Chain reactions of transaction issues
As I write this post, $800 in debts we owe to our customers is stuck in transaction hell on the bitcoin blockchain. This money was withdrawln from Yobit two days ago, and never confirmed because the transaction fee Yobit paid was only 41 cents. As a result, our customers didn't get paid. This is an example of a new effect that we see more and more often - a chain reaction of events, each of which is made worse by software systems that are not programmed to handle these delays.
The problem with the bitcoin network is that it is not possible for a recipient to increase the fee of a transaction that he wants to receive. Even if we wanted to do so, we could not pay $1.00 to force this Yobit transaction through, no matter how our software is designed. Plus, there is a huge design flaw in bitcoin: there is no way for Yobit to know when it is even necessary to resend this transaction. But let's assume that we actually do get the withdrawal eventually.
In the past three years, we have never lost even a single cent due to network errors. The reason is straightforward: we begin a database transaction, subtract the payouts from user balances, allow CHECK constraints to run to ensure that nobody's balance goes under zero due to a bug, prepare the network transaction (rollback if failed), send the network transaction (rollback if failed), and then once success is achieved, commit the database transaction. Then, we repeat this procedure for every other coin. When the network functions properly, this procedure guarantees consistency between a coin network and the database. If any errors occur, a rollback happens and the database balances remain unchanged.
But when transactions don't confirm on a network, the database balances end up out of sync with the wallet balances. This leads to a chain reaction of failures. Look at the options we have now, all of which are horrible:
3. Increased security issues
One of the worst effects of the network dysfunction is that the increased payout thresholds reduce our security.
The simplest, and most effective, security measure that we take is to hold as little money as possible. This is why we have payout thresholds in the first place; hackers have little incentive to spend weeks trying to find security vulnerabilities, and risk a prison sentence, for less than $10,000. There are plenty of exchanges out there that have hot wallets containing millions of dollars, so while we take security seriously, the low amount of money at stake reduces our appeal to criminals even if we had poor security.
Unfortunately, the feasibility of retaining small sums of money is dwindling because we can't pay people as often and can't take profit as often. We now sell our bitcoins in batches of $2,500 or so, whereas I can recall executing sales to take profit every $1,000 in the past. Selling in larger batches reduces the number of transactions we need to execute to get rid of these bitcoins.
The days when people would move small amounts of money to offline cold storage are gone. As time goes on, bitcoins become less mobile, and there are fewer incentives to keep the absolute minimum amount of money accessible online. This is likely to lead to more thefts and hacking incidents.
4. Wasted development time
Some Bitcoin issues cannot be easily worked around and it is now necessary to waste time fixing them. Here is a list of issues that take hours of our time that are directly caused by the transaction limit and which are all new problems:
5. Holding more coins in exchanges
Bitcoins on the network are less valuable than those in exchanges, because some exchanges have no withdrawal fees (like Coinbase) or have not raised their withdrawal fees in a long time and the network is now causing them to lose money (like Poloniex). By holding bitcoins in exchanges, we can save money.
This is why we only withdrawal from exchanges when we need to pay customers. It is also why we run a surplus of several thousand dollars. Because the surplus is spread across exchanges, when there is more money any particular day's payout is likely to require the full balance of fewer exchanges to satisfy it. If we had no surplus, we would have to withdrawal all money from all seven exchanges every day; with a $2000 surplus, we can often save money by withdrawing from just two exchanges.
This fact will horrify the group of people who subscribe to the mantra of "if the bitcoins aren't in your wallet, you don't own them." Unfortunately, times have changed.
6. The definition of "theft"
While I sincerely hope that it doesn't come to this, we may soon have the opportunity to set a precedent on a groundbreaking incident of the definition of "theft" in Bitcoin.
The Bitcoin network has become so dysfunctional that exchanges are not only getting delayed with withdrawals, but now one particular exchange has sent a transaction with a fee so low that it was not mined. Four days later, it is still in the memory pool with zero confirmations. Since this money is owed to customers, bitcoin payouts are being delayed every day until enough new money is mined to cover this lost customer debt.
For now, we've submitted two friendly customer support tickets with the exchange and are hoping that they will finally reply. However, our third contact with them will be an E-Mail, and then after that, a call, stating that we will consider their actions as theft should they not resend the withdrawal with a higher fee.
If this issue goes to court, which I hope is not necessary, then it will be the first time to my knowledge that a government would define what blockchain theft is. Is a transaction on the blockchain with insufficient fees a valid payment that satisfies a debt? If so, then the usefulness (and therefore price) of bitcoin would fall dramatically. If it is not, what if there is a transaction that has sufficient fees but then the cost of transactions rises between sending time and mining time? Would a ruling that the exchange stole from us result in an increase of transaction fees across the board, as businesses try to protect themselves from liability? Or, would such a ruling also crash the price of bitcoin, as businesses turn away from the blockchain on the grounds that the risk of legal action is too high to use bitcoins in commerce?
----------------
In conclusion, the time has come for us to do something about these issues and to be proactive for when they worsen in the future. Now that major development on X11 mining is finished, after Chris releases the test server one of my goals for the weekend of March 10-11 is going to be to reduce the system's reliance on Bitcoin, wherever possible.
The first step is simple: for people who want to be paid in dollars, we can send them ETH and use Coinbase's API to sell ETH instead of BTC.
I've also come up with a plan for how we can save money in both steps of the payout process. This plan involves purchasing ETH in all exchanges immediately before bitcoin payouts are due, withdrawing the ETH to the GDAX, converting it to BTC there, and then using Coinbase to send out the transactions because they pay the fees. Since network conditions have reduced customer demand for bitcoin to just $1000 in BTC per day in payouts now, the fees to us for doing this are the following:
Hopefully, this article has demonstrated that the Bitcoin network is now the domain of gambling speculators, useless for commerce, and people like us have been and plan to spend hundreds of hours to work around these issues. What should concern investors who are pouring money into bitcoins and this ETF is that once we write this code, we aren't going to go back to the old way of doing things, ever. ETH's transaction fees are nearly zero, so even if BTC's transaction fees fell to zero, it would not make economic sense to assign a developer to modify a tested system to use BTC again. The only way that reverting would make sense is if the fees in ETH rose so high in comparison to BTC that we could save the thousands of dollars it costs to take the risk of new code and pay someone to write it, and that is unlikely to happen. BTC's fees falling to 10% below those of ETH will not be sufficient to revert once this is done.
So as people blindly celebrate the new Bitcoin all time high, I'll be continuing to program our systems to work around BTC's useless network using Ethereum.
Not all of these effects are obvious and some will likely surprise people who believed the stalemate would result in different outcomes.
-------------------
1. Changes in payout requests
Transaction fees are bad for business and complex to users, which is why corporations like Coinbase hide them. At Prohashing, we also aim to ensure that users don't have to worry about transaction fees if they don't want to. To accomplish that objective, we pay users for free after their balances rise above a threshold amount in their requested currencies. This "payout threshold" is targeted so that 0.5% of the payout will be lost to fees. The payout threshold used to be a flat $1 for all coins. It remains $1 for all coins except Bitcoin, which was raised to $6, $12, and now stands at $20 (viewtopic.php?f=4&t=1095), because our transactions now cost about $3.10 and there are about 40 outputs.
Each time we raised the threshold, we expected that people would either choose to pay the fee at the prior threshold, stop mining with us, or simply allow us to hold more of their money and get paid less frequently in larger amounts. Instead, what we've found is that the payout thresholds have led to a significant decrease in the usage of Bitcoin. In fact, an examination of the "Payout coins owed" chart at https://prohashing.com demonstrates that customers requesting payment in bitcoins often falls into the minority (today appears to be an abnormality, because one large miner switched his payout coin apparently due to the bubble).
When I asked some larger customers why they changed currencies, the answer was straightforward, but obvious in retrospect. Waiting for larger payouts is not a big deal to them. The problem is that they don't want their money "stuck" on a non-functional network. To see why, consider that for every transaction we sent to a person, he or she has to pay a much greater amount to transfer the money to someone else, because there is a set overhead to transactions. Since money is useless unless it can be spent, the value of each output on the Bitcoin blockchain is reduced by an amount of at least (2 * current transaction fee). That pays to ignore the threshold and then pay fees again to spend the money.
That might be the end of it, except that it's very unlikely that the bitcoins are only penalized by a multiple of the current fees. The output isn't likely to be spent today; it is likely to be spent days, months, or years from now. Let's assume that median fees rise to $20.00 by the end of the year, which I consider to be likely given the current rate of growth. Our customers recognize that their $20.00 payouts today will have no value whatsoever by the end of the year, so they are choosing not to be paid in bitcoins.
This is why fewer than 50% of people now request bitcoins as payments. Altcoins that used to be considered minor players, like Gamecredits and Gulden, are quickly picking up the slack, and Litecoins, which have proven to be a much more stable store of value than bitcoins over the past few months, are gaining traction as well.
2. Chain reactions of transaction issues
As I write this post, $800 in debts we owe to our customers is stuck in transaction hell on the bitcoin blockchain. This money was withdrawln from Yobit two days ago, and never confirmed because the transaction fee Yobit paid was only 41 cents. As a result, our customers didn't get paid. This is an example of a new effect that we see more and more often - a chain reaction of events, each of which is made worse by software systems that are not programmed to handle these delays.
The problem with the bitcoin network is that it is not possible for a recipient to increase the fee of a transaction that he wants to receive. Even if we wanted to do so, we could not pay $1.00 to force this Yobit transaction through, no matter how our software is designed. Plus, there is a huge design flaw in bitcoin: there is no way for Yobit to know when it is even necessary to resend this transaction. But let's assume that we actually do get the withdrawal eventually.
In the past three years, we have never lost even a single cent due to network errors. The reason is straightforward: we begin a database transaction, subtract the payouts from user balances, allow CHECK constraints to run to ensure that nobody's balance goes under zero due to a bug, prepare the network transaction (rollback if failed), send the network transaction (rollback if failed), and then once success is achieved, commit the database transaction. Then, we repeat this procedure for every other coin. When the network functions properly, this procedure guarantees consistency between a coin network and the database. If any errors occur, a rollback happens and the database balances remain unchanged.
But when transactions don't confirm on a network, the database balances end up out of sync with the wallet balances. This leads to a chain reaction of failures. Look at the options we have now, all of which are horrible:
- We could resend the original transaction at a different fee the next day, after adjusting for the new fee. If we do this, nothing else can be done until we determine which of the two will confirm, and adjust the balances accordingly due to which fee was charged. We can't send the next day's payouts until both the previous day's transaction confirms, and the database is updated with the corrections.
- We could add yesterday's payouts to today's payouts and send them both, at a higher fee. If we do this, then the previous day's transaction could still confirm in the meantime, leaving us to do complex math to figure out how much people are still owed.
- We could issue a "child pays for parent" transaction, where the next day's fees are much higher, but most wallets don't support this and because many pools don't either, the fee for this transaction has to be even higher than normal.
3. Increased security issues
One of the worst effects of the network dysfunction is that the increased payout thresholds reduce our security.
The simplest, and most effective, security measure that we take is to hold as little money as possible. This is why we have payout thresholds in the first place; hackers have little incentive to spend weeks trying to find security vulnerabilities, and risk a prison sentence, for less than $10,000. There are plenty of exchanges out there that have hot wallets containing millions of dollars, so while we take security seriously, the low amount of money at stake reduces our appeal to criminals even if we had poor security.
Unfortunately, the feasibility of retaining small sums of money is dwindling because we can't pay people as often and can't take profit as often. We now sell our bitcoins in batches of $2,500 or so, whereas I can recall executing sales to take profit every $1,000 in the past. Selling in larger batches reduces the number of transactions we need to execute to get rid of these bitcoins.
The days when people would move small amounts of money to offline cold storage are gone. As time goes on, bitcoins become less mobile, and there are fewer incentives to keep the absolute minimum amount of money accessible online. This is likely to lead to more thefts and hacking incidents.
4. Wasted development time
Some Bitcoin issues cannot be easily worked around and it is now necessary to waste time fixing them. Here is a list of issues that take hours of our time that are directly caused by the transaction limit and which are all new problems:
- Exchange customer service requests: When transactions get stuck, a support ticket needs to be generated by us at the exchange. These tickets are often ignored or take days to be resolved with multiple replies from both parties.
Disabling exchanges: When an exchange fails to fix bitcoin transaction issues, we have to disable trading at that exchange so that no more money flows in until we can get a guarantee the issue is resolved. This takes a long time and reduces customers' profits from mining coins where that exchange has the highest sell prices.
Responding to complaints: We repeatedly have to reply to customers in our forums and in E-Mails who complain about payout delays. These people are right to be angry, but we still have to reply to apologize and tell them that their money will be available later.
Correcting balances: When transactions get stuck, we need to correct people's balances to reflect what actually happened. A secondary effect is that customers notice they have more or less money and ask why, requiring additional time to explain the reasons.
Computing and adjusting fees: Determining what people are willing to pay and how we can make a profit to set our payout thresholds is not a simple task. And again, each rise in the payout threshold generates customer service E-Mails.
5. Holding more coins in exchanges
Bitcoins on the network are less valuable than those in exchanges, because some exchanges have no withdrawal fees (like Coinbase) or have not raised their withdrawal fees in a long time and the network is now causing them to lose money (like Poloniex). By holding bitcoins in exchanges, we can save money.
This is why we only withdrawal from exchanges when we need to pay customers. It is also why we run a surplus of several thousand dollars. Because the surplus is spread across exchanges, when there is more money any particular day's payout is likely to require the full balance of fewer exchanges to satisfy it. If we had no surplus, we would have to withdrawal all money from all seven exchanges every day; with a $2000 surplus, we can often save money by withdrawing from just two exchanges.
This fact will horrify the group of people who subscribe to the mantra of "if the bitcoins aren't in your wallet, you don't own them." Unfortunately, times have changed.
6. The definition of "theft"
While I sincerely hope that it doesn't come to this, we may soon have the opportunity to set a precedent on a groundbreaking incident of the definition of "theft" in Bitcoin.
The Bitcoin network has become so dysfunctional that exchanges are not only getting delayed with withdrawals, but now one particular exchange has sent a transaction with a fee so low that it was not mined. Four days later, it is still in the memory pool with zero confirmations. Since this money is owed to customers, bitcoin payouts are being delayed every day until enough new money is mined to cover this lost customer debt.
For now, we've submitted two friendly customer support tickets with the exchange and are hoping that they will finally reply. However, our third contact with them will be an E-Mail, and then after that, a call, stating that we will consider their actions as theft should they not resend the withdrawal with a higher fee.
If this issue goes to court, which I hope is not necessary, then it will be the first time to my knowledge that a government would define what blockchain theft is. Is a transaction on the blockchain with insufficient fees a valid payment that satisfies a debt? If so, then the usefulness (and therefore price) of bitcoin would fall dramatically. If it is not, what if there is a transaction that has sufficient fees but then the cost of transactions rises between sending time and mining time? Would a ruling that the exchange stole from us result in an increase of transaction fees across the board, as businesses try to protect themselves from liability? Or, would such a ruling also crash the price of bitcoin, as businesses turn away from the blockchain on the grounds that the risk of legal action is too high to use bitcoins in commerce?
----------------
In conclusion, the time has come for us to do something about these issues and to be proactive for when they worsen in the future. Now that major development on X11 mining is finished, after Chris releases the test server one of my goals for the weekend of March 10-11 is going to be to reduce the system's reliance on Bitcoin, wherever possible.
The first step is simple: for people who want to be paid in dollars, we can send them ETH and use Coinbase's API to sell ETH instead of BTC.
I've also come up with a plan for how we can save money in both steps of the payout process. This plan involves purchasing ETH in all exchanges immediately before bitcoin payouts are due, withdrawing the ETH to the GDAX, converting it to BTC there, and then using Coinbase to send out the transactions because they pay the fees. Since network conditions have reduced customer demand for bitcoin to just $1000 in BTC per day in payouts now, the fees to us for doing this are the following:
- Buy ETH at another exchange: 0.25% or $2.50
- Withdrawal the ETH - negligible, perhaps $0.10
- Sell ETH at the GDAX: 0.25% or $2.50
- Pay customers in BTC: zero
Hopefully, this article has demonstrated that the Bitcoin network is now the domain of gambling speculators, useless for commerce, and people like us have been and plan to spend hundreds of hours to work around these issues. What should concern investors who are pouring money into bitcoins and this ETF is that once we write this code, we aren't going to go back to the old way of doing things, ever. ETH's transaction fees are nearly zero, so even if BTC's transaction fees fell to zero, it would not make economic sense to assign a developer to modify a tested system to use BTC again. The only way that reverting would make sense is if the fees in ETH rose so high in comparison to BTC that we could save the thousands of dollars it costs to take the risk of new code and pay someone to write it, and that is unlikely to happen. BTC's fees falling to 10% below those of ETH will not be sufficient to revert once this is done.
So as people blindly celebrate the new Bitcoin all time high, I'll be continuing to program our systems to work around BTC's useless network using Ethereum.