Chris makes breakthrough on orphan rate
Posted: Fri Jan 09, 2015 11:56 am
I thought I would share a breakthrough that Chris made on the orphan rate of coins yesterday. This breakthrough is likely to significantly increase profitability when we release its fix this weekend.
Ever since the pool was started, Chris noticed that we tended to make more profits than expected. We thought that we were just lucky, because after examining a lot of code we never were able to determine why the numbers always seemed to favor us. The payouts being displayed on the site and reported to PoolPicker were correct, and people were being paid what was promised. But those payouts were still lower than we thought that they should be.
It turns out that the orphan penalty being applied to coins is far too high. When this issue is resolved, it has the potential to significantly change not only how much is made on the coins we mine, but also the coins we choose to mine themselves. We finally came to this issue when we evaluated how Nyancoins were being mined. In the case of Nyancoins (and other coins), we compute a penalty to account for orphaned blocks. The penalty is the total of the number of orphaned blocks divided by the total of (confirmed plus orphaned blocks). Unconfirmed blocks are not included in the current formula.
The problem is that we almost always know whether a block is orphaned immediately, but the network doesn't allow blocks to be confirmed for many blocks. For bitcoins, 101 blocks are required for confirmation, but if we submitted an orphaned block to the network, the daemon is likely to state that it was orphaned within 0.25s of the submission.
Consider if we look back 10 blocks to compute this penalty and see this:
*Unconfirmed
*Orphaned
*Unconfirmed
*Unconfirmed
*Orphaned
*Unconfirmed
*Confirmed
*Confirmed
*Orphaned
*Confirmed
The current orphan rate would be 50% (3 orphaned divided by the 6 total blocks that are not unconfirmed). This number would be correct if the probability that the unconfirmed blocks are going to be orphaned is also 50%. But that probability of unconfirmed blocks later being orphaned is close to zero instead, because of what we talked about above.
The correct solution is to consider unconfirmed blocks as almost certain to be confirmed, since there has rarely or never been a case where those blocks later show up as orphaned. If we now change the formula to consider "unconfirmed" as "confirmed," then we get an orphan rate of 30% (3 orphaned divided by the total number of blocks, which is 10).
As you can see, that 20% lower orphan rate leads to a 40% increase in profitability. If the current profitability is 0.0002, that means that miners should expect to earn 0.00028 instead. We don't know what the actual increase will be until we release the fix over the weekend. In fact, some coins would be completely skipped because the orphan rate for those coins was over the set threshold, causing suboptimal choices.
Even after the release, the numbers on the site will take a day to reflect the latest profitability, so we'll see what happens by Monday. Miners will be paid more immediately following this release, but how much more won't be known until we discover the new behavior of the algorithm.
Ever since the pool was started, Chris noticed that we tended to make more profits than expected. We thought that we were just lucky, because after examining a lot of code we never were able to determine why the numbers always seemed to favor us. The payouts being displayed on the site and reported to PoolPicker were correct, and people were being paid what was promised. But those payouts were still lower than we thought that they should be.
It turns out that the orphan penalty being applied to coins is far too high. When this issue is resolved, it has the potential to significantly change not only how much is made on the coins we mine, but also the coins we choose to mine themselves. We finally came to this issue when we evaluated how Nyancoins were being mined. In the case of Nyancoins (and other coins), we compute a penalty to account for orphaned blocks. The penalty is the total of the number of orphaned blocks divided by the total of (confirmed plus orphaned blocks). Unconfirmed blocks are not included in the current formula.
The problem is that we almost always know whether a block is orphaned immediately, but the network doesn't allow blocks to be confirmed for many blocks. For bitcoins, 101 blocks are required for confirmation, but if we submitted an orphaned block to the network, the daemon is likely to state that it was orphaned within 0.25s of the submission.
Consider if we look back 10 blocks to compute this penalty and see this:
*Unconfirmed
*Orphaned
*Unconfirmed
*Unconfirmed
*Orphaned
*Unconfirmed
*Confirmed
*Confirmed
*Orphaned
*Confirmed
The current orphan rate would be 50% (3 orphaned divided by the 6 total blocks that are not unconfirmed). This number would be correct if the probability that the unconfirmed blocks are going to be orphaned is also 50%. But that probability of unconfirmed blocks later being orphaned is close to zero instead, because of what we talked about above.
The correct solution is to consider unconfirmed blocks as almost certain to be confirmed, since there has rarely or never been a case where those blocks later show up as orphaned. If we now change the formula to consider "unconfirmed" as "confirmed," then we get an orphan rate of 30% (3 orphaned divided by the total number of blocks, which is 10).
As you can see, that 20% lower orphan rate leads to a 40% increase in profitability. If the current profitability is 0.0002, that means that miners should expect to earn 0.00028 instead. We don't know what the actual increase will be until we release the fix over the weekend. In fact, some coins would be completely skipped because the orphan rate for those coins was over the set threshold, causing suboptimal choices.
Even after the release, the numbers on the site will take a day to reflect the latest profitability, so we'll see what happens by Monday. Miners will be paid more immediately following this release, but how much more won't be known until we discover the new behavior of the algorithm.