Status as of Tuesday, September 12, 2017

[Steve Sokolowski]

Here's a brief update today:

• Chris released the latest version of the mining server code last night, so all features we talked about are available now, except case-insensitive usernames. We pulled those because we need to do more research as to whether lowercasing every username every time someone submits a share will impact performance.
• We haven't been paying much attention to memory usage, because memory can be dealt with by spending money. However, I think we can get memory usage down by a lot, and this release is 50% more efficient than the last one. The next round of performance improvements will be spent on conserving memory, on the theory that garbage collection takes less time when less memory is allocated, and that frequently used data can be put in the processors' caches and there will be fewer cache misses.
• Bringing the system back online took longer than expected last night because the hypervisor ran out of memory. That happened because we had moved the main disk for the mining server to a ramdisk to eliminate the network disk we were using before. The elimination of the network attached storage configuration for that one computer, which we thought would be fine because the mining server does not need a disk, seems to have reduced disconnects to miners. Apparently, Debian hangs when it is overloaded with disk accesses, and that doesn't happen when a non-network disk is used. While those disconnect issues are done with, the downside is that we will have to restart the mining server frequently for the next few days until we can install a normal solid state disk in the server. The solid state disk won't arrive until Friday.
• Chris also discovered that DNS lookups can cause freezeups as well. He installed a DNS server on the VPN, so that DNS lookups that were already performed for other services can be used by all the virtual machines inside the network. That seems to have improved network connectivity even further.
• Now that performance is under control for a while, our focus will move towards reliability. Our first step will be software to automatically restart things when they crash. Most of the crashes - the website, for example - occur in third-party software like Tomcat, and these are not as easy to debug as they would be if they happened in our own code. Once auto-restarts are in place, Chris will then move to try to prevent the crashes at all.

[gjaramillo]

thanks steve

[simonjbcmm]

Thank you Steve and Chris for all you do. Your PROHASHING service is fantastic!

With appreciation,
Jonathon

[mickeekung]

Thanks. Additionally, Any plan or news about removing ignored diff function?

[FRISKIE]

@ Steve - so many of the things you guys have discovered with regards to network and systems infrastructure, and with virtualized infra is so basic for those of us who work in that area of technology, that it begs the question of why you guys insist on stumbling your way forward re-inventing the wheel each time, rather than spend for a consultant.

Surely all the money lost by now could have paid for an army of consultants by now.