A few thoughts - Friday, August 8, 2014

[Steve Sokolowski]

Good afternoon! A few thoughts for lunch today:

Gavin Andresen makes breakthrough?

Today, /u/gavinandresen issued a tweet suggesting that he has discovered a more efficient method of storing transactions. He didn't provide many details in the tweet, but the way most users in /r/bitcoin interpreted it has significant implications.

The issue goes back to a major weakness in the bitcoin protocol that has been present since it was designed. During mining, transactions are being sent across the network, eventually reaching all other nodes, without being included in blocks yet. Miners need to know which transactions are available to be able to select some to put into blocks. When a block is found, then the hash of the found block is transmitted, along with some of the transactions that most nodes already have in memory. Re-sending transactions that already exist is unnecessary because the other nodes can pick which transactions to include in the block if they are simply told which to include, rather than being sent a copy.

In the simplest enhancement to resolve this problem, nodes would just send a pointer to the transactions that should be included in a block, rather than the transactions themselves. Large transactions with many inputs and outputs can be reduced to a single hash value, which is then sent by the miner. The other nodes would then construct the mined block by looking up the hash value and inserting the corresponding transaction. However, there are some issues with this implementation that are out of the scope of this comment.

It appears that something better has been accomplished. While it is unclear exactly what Andresen has done, the consensus seems to be on a solution that could reduce the network load and disk utilization significantly. Instead of bandwidth increasing linearly with the number of transactions, the amount of bandwidth required to notify other nodes that a block has been found would be nearly constant. This is significant because a 10kb block would require the same amount of bandwidth to upload as a 1MB block would. Since miners lose money due to orphaned blocks when it takes a long time to upload large blocks with many transactions, currently many miners only include transactions with large fees (or no transactions at all).

But if the resources were constant to include any number of transactions, then most miners would include as many transactions as possible in their blocks. After all, doing so wouldn't increase orphan rate whatsoever, and there would be no minimum fee that would be required to make the bandwidth spent on the transactions worth the additional risk of orphange. Transaction fees would decline significantly, because miners might as well fill up blocks with zero-fee transactions at no extra cost. Doing so increases the usefulness of the network because people's money moves more quickly and more cheaply, increasing the amount of money people are willing to pay for bitcoins, which means that miners make more money from mining.

Andresen did not say that he had actually implemented this functionality in his tweet, so here's to hoping he'll publish his findings so that work can begin. This is the sort of technological advance that will be necessary to jump-start bitcoins out of the slump they're going to be in for the next few months.


Derivatives and futures markets ramping up

One of the more interesting developments recently is the advent of bitcoin derivatives and futures contracts. Chinese exchanges have reportedly been deluged with investors wanting to hedge their holdings in bitcoin futures.

In other markets, short-selling, futures, and derivatives are the products that stabilize prices. By allowing people to profit from downswings as well as upswings in the currency, crashes don't necessarily become self-sustaining. With short selling, there is always someone making money during a crash, and that person has the opposite incentives than do the people who are buying and holding.

Futures contracts further stabilize prices by allowing people to bet on the price of bitcoins at points in time long from now. Imagine that your company needs bitcoins two months from now, and you think the price of bitcoins is going to fall. Normally, you would wait until the day before to buy those bitcoins, since you think they will be cheaper. However, there could be a bubble instead, causing huge losses for you. Therefore, you sign a contract to buy bitcoins in two months at some agreed price. If the market thinks that there will be a fall in price, then you will get some savings below market rate, but not as much as if you had waited and bought in two months. However, since a bubble in the interim is irrelevant to you, you run zero risk of bankruptcy if you buy such a contract now.

People have long been saying that as soon as these types of financial products were available for bitcoins, the price would stabilize. I don't think that many people predicted how quickly that would happen, though. Volatility isn't anywhere close to that of a year ago. While a high price will certainly stabilize the value of bitcoins further, it may be true that the peaks and valleys will smooth out more quickly than anyone thought. It may also be worth considering that the changing fundamental that broke the bubble cycle could be the availability of these new products. These weren't available during any of the previous cycles and represent a significant paradigm shift in how bitcoins are traded.

Keep in mind that futures are just getting started; the markets only opened a week ago. As the markets get larger and spread to more exchanges, the stability effect would be expected to increase.


Do mining pools lose money because they don't understand their own software?

For the past 20 days, I've been working on one of the last issues with the altcoin mining pool. Mining should be a very regular business; it is possible to compute exactly what you should be earning at any given point in time. Therefore, you can produce charts every day and analyze how much your actual earnings deviate from the expected values. You can see one of these charts at http://shoemakervillage.org/temp/profit2014-08-08.jpg.

When we started this investigation, the pool was losing about 40%, and now we are almost there - a little below the expected profit (which should be 5% for this testing phase). The reasons why ranged from things like how orphaned blocks were calculated, differences in the way coin daemons work, race conditions in threads, deadlocks, incorrect calculation of transaction fees, not practicing selfish mining, having wrong constants in formulas, coins not appearing in exchanges, transfer fees being higher than expected, and at least 20 other issues.

When we looked at other pools to determine how they dealt with these issues, they never mention any of these things. They seem to just pay out what they earn. Is it possible that most miners don't even bother to check whether what they are doing is bugged or optimal? If so, that would be a shocking finding. How can someone operate a business without even knowing whether the operation is losing money?


Transaction watch
---
The number of transactions per day (https://blockchain.info/charts/n-transa ... 0&address=) continues to rise for the third consecutive reporting period. There are now far more transactions than there have been at any time in the past few months.

This event is pretty interesting for several reasons. First, it is coming at a time when the hashrate of the network is not dramatically increasing like it has in the past, so blocks are occurring less frequently, but there are more transactions anyway. The average size of blocks is also increasing significantly. Whereas earlier in the month it was common to see blocks as small as 52kB, blocks are frequently approaching 500kB. I'm hoping that someone who has more available time than I do produces a report to see whether there are more dust transactions being included, or whether this chart represents a true increase in usage of the network.


Mining crash underway

/u/bit_by_bit has some analysis that shows that the price of mining contracts at CEX has dropped from $2.75 to $2.03 over the course of a few weeks. This must be the mining bubble I mentioned a while ago coming to an end.

I personally thought that the bubble would crash harder, leaving the market with a period of decreasing difficulty and block times well over ten minutes, but I also don't think that the price of bitcoins has reached the bottom yet. I imagine we will see a period of decreasing difficulty at some point before the next rise starts to occur.

In the past, there was always profit in bringing more equipment online as soon as possible, since the price rose more quickly than mining equipment could be produced. With miners having stabilized at equilibrium now, consider a scenario where there is a flash crash in bitcoin price. That would likely lead to a mining crash, since miners are just barely breaking even now. In a feedback loop, the reduced usefulness of the network (because block times are longer and fewer transactions can be processed) leads to prices falling further, causing more miners to shut down, and so on. The same thing would happen, for example, if the 1MB transaction limit were reached.

While I don't agree with /u/bit_for_bit's idea that mining can influence price increases, I do think there's something to be said about a mining crisis initiating a crash.


Other

* **Days until the comment period ends: 31**