Status as of Sunday, May 12, 2019
Posted: Sun May 12, 2019 10:28 am
Good morning! Today, I'm going to focus entirely on the mining server issues I've been investigating.
- There are a few core problems, all of which may be related. The first is that the server was incorrectly crediting miners for stale shares where merge mined blocks were found. That issue was corrected, and resulted in a significant drop in profitability for about 1 in 1000 miners who were submitting these shares over and over due to misconfigured miners. I evaluated the impact on other miners and it was less than 0.012% in earnings, so we took no additional action so as not to accidentally introduce a new bug that we might not be aware of.
- The second issue is that luck seems to have a higher variance on scrypt than on other algorithms. This issue was not detectable until we were able to finally develop a query that eliminated the effects of other non-luck effects on revenue. The query is currently showing that DigiBytes have luck around 1.06, while Graviocoins have luck at 0.94. The probability of these figures being so far away from 1 after 6 days of mining is extremely low, given that we find so many of these blocks. On X11, however, Dash has luck of 1.002 despite many fewer blocks being found. We aren't sure whether this issue is indicative of a bug, or whether the query is incorrect. I plan to continue an investigation into this after the holiday in June.
- The third issue is that a significant number of miners are submitting blocks that are far behind the current block. Some miners are 4 or 6 or even 27 blocks behind the current block. The issue seems to occur most with DigiBytes, but that might simply be because the network is so fast that if whatever causes the problem requires a set time to pass, more blocks happen during that time. This is the issue that is the most significant and which I've spent about 60 hours investigating. If a solution is found, then profitability will rise by 9%.
- I looked through code for many days and found a lot of bugs, but none were the cause of this problem. Today, I'm going to take a new path and release a new version to one of the mining servers that attempts to detect when the problem occurs and sends new work to those miners. If the miners respond and the number of blocks behind decreases, then it should be possible to mitigate the problem significantly even if the cause is never found. If the miners do not respond and keep submitting stale shares, then that may indicate that there is a fundamental misunderstanding of the way the miners respond to work restarts. We should know by this evening which issue is the case.