System issue

News updates about the Prohashing pool
Forum rules
The News forum is only for updates about the Prohashing pool.

Replies to posts in this forum should be related to the news being announced. If you need support on another issue, please post in the forum related to that topic or seek one of the official support options listed in the top right corner of the forums page or on prohashing.com/about.

For the full list of PROHASHING forums rules, please visit https://prohashing.com/help/prohashing- ... rms-forums.
Locked
User avatar
Steve Sokolowski
Posts: 4585
Joined: Wed Aug 27, 2014 3:27 pm
Location: State College, PA

System issue

Post by Steve Sokolowski » Wed Feb 03, 2016 9:20 am

Hi,

There's some sort of system issue that I'm told took the mining server offline. Unfortunately, I haven't been able to get in contact with Chris right now.

I'll keep trying to wake him up. Look here for more information as it becomes available.
User avatar
Steve Sokolowski
Posts: 4585
Joined: Wed Aug 27, 2014 3:27 pm
Location: State College, PA

Re: System issue

Post by Steve Sokolowski » Wed Feb 03, 2016 12:11 pm

The system has been brought back online by Chris. He will first execute yesterday's payouts, and will then determine how many shares, if any, were lost, and credit balances. Then, he'll try to figure out what happened.

He should be able to make a post here this afternoon with more info.
User avatar
Chris Sokolowski
Site Admin
Posts: 945
Joined: Wed Aug 27, 2014 12:47 pm
Location: State College, PA

Re: System issue

Post by Chris Sokolowski » Wed Feb 03, 2016 9:24 pm

Thanks everyone for your patience during the unexpected downtime. The mining server crashed around 2:00 AM EST, but even before that, it was queueing shares for hours before that. The last share inserted was at 12:45 AM EST, meaning the mining server was 75 minutes behind at the time of the crash.

The cause of the crash was that the mining server was overloaded. Every time that one of the mining-enabled coins has a new block, the mining server re-analyzes the profitability of that coin, computes the optimal assignment of workers to all coins, and then re-assigns workers. As we have been adding coins over the past few weeks, the frequency at which one of the mining coins has a new block has been increasing, and the number of workers to assign has also increased. Last night, the server began receiving notifications of new blocks faster than it could compute the optimal assignments of miners, and as a result it crashed.

Steve has prepared a mining server update the reduces load by ignoring the step of analyzing worker assignments when we receive a new block on a coin that is less profitable than Litecoin. This single change should reduce the number of worker assignment computations by 95%. I will be testing and releasing this code tonight, which may result in a 30 second disconnect.

As compensation for the crash and missing shares, I have increased user balances by adding 1.75 hours of mining time to all users balances, which would be the earnings as if the mining server had crashed at 2:30 AM. All users mining between 9:45 PM and 12:45 AM EST yesterday received extra money proportional to their hashing power during that time.

As always, please fee free to comment if you have any questions. Thanks again for your support.
User avatar
Chris Sokolowski
Site Admin
Posts: 945
Joined: Wed Aug 27, 2014 12:47 pm
Location: State College, PA

Re: System issue

Post by Chris Sokolowski » Fri Feb 05, 2016 2:09 am

It looks like the server still isn't performing up to our expectations. The 35 GH/s spike earlier today caused the server to get overloaded, and while no shares were lost, the server did get behind in inserting shares. I am fairly confident this is not a database issue but rather a mining server issue. We are working to profile performance and resolve bottlenecks. Over the next few days, you can expect a 30 second restart of the mining server every morning to deploy optimizations.
Locked