System to be offline Tuesday, November 24
Posted: Sat Nov 07, 2015 8:17 am
We previously announced that we would be taking some major downtime during the holidays, and we can now announce that we will be taking the pool offline on Tuesday, November 24. If (as rootdude knows) there is an unforeseen problem, the downtime could last into some of the next day. We will probably start at 9:00am EST.
The major purpose of this downtime is to upgrade the system to Postgres 9.5. Postgres 9.5's INSERT ON CONFLICT statement is so important that we will upgrade anyway even if 9.5 is still in a beta version then. The performance increase provided by INSERT ON CONFLICT is about double that of "IF NOT FOUND THEN... INSERT" statements, and it also prevents concurrency errors, allowing us to multiprocess all of the share insert statements so that we can run 24 cores of parallel inserts, rather than the one we are limited to now. The mining server code using Postgres 9.5 is tested already and will be released after we migrate the data.
We have 1TB of data to migrate, so simply copying the data around will be time-consuming. During this downtime, we will also remove the journaling filesystem on the database server, because it is not necessary for Postgres and adds additional overhead. A few other points:
The major purpose of this downtime is to upgrade the system to Postgres 9.5. Postgres 9.5's INSERT ON CONFLICT statement is so important that we will upgrade anyway even if 9.5 is still in a beta version then. The performance increase provided by INSERT ON CONFLICT is about double that of "IF NOT FOUND THEN... INSERT" statements, and it also prevents concurrency errors, allowing us to multiprocess all of the share insert statements so that we can run 24 cores of parallel inserts, rather than the one we are limited to now. The mining server code using Postgres 9.5 is tested already and will be released after we migrate the data.
We have 1TB of data to migrate, so simply copying the data around will be time-consuming. During this downtime, we will also remove the journaling filesystem on the database server, because it is not necessary for Postgres and adds additional overhead. A few other points:
- It is likely that the downtime will last a minimum of twelve hours.
- Most services will be unavailable during this time: mining, trading, block explorers, daemons, and the website. The forums will remain online, and customer service E-Mails will still be answered on a delay.
- The block explorers can be expected to be out of date until after Thanksgiving, because they will need to catch up scanning the chains after the system comes back online.
- The database will be "behind," with statistics being delayed for much of Monday, November 23, as we delete unnecessary data from the tables in preparation for this upgrade. However, mining will continue as normal during that time.
- Payouts for mining conducted on Monday will be paid on Wednesday instead of on Tuesday. If the shutdown lasts longer than expected, Chris will run a manual payout whenever the system returns.