Website issues

[Steve Sokolowski]

We're encountering another round of those invalid password requests, like what happened a few weeks ago. Every time the pattern of these IP addresses changes, we need to make a change to the detection software. Making a change is trivial, but it takes time to catch up with the bans.

So far, the system has processed 237,000 bad IP addresses and continues to ban more. We expect the website to return to normal in an hour. Mining is not affected and no money is lost.

[Steve Sokolowski]

These issues appear now to have gotten below the threshold where the site is slow. More IP addresses will continue to be banned - 10,000 additional ones were banned since the last post, and the site will become even faster as time goes on.

[Steve Sokolowski]

A further update: I modified the software that bans IP addresses to process the addresses more quickly, and we also expanded the number of cores on the webserver from 5 to 10. 120,000 additional IP addresses have been banned since yesterday, bringing the total to 345,000.

Tonight, the website will be taken offline for a short period while we move it to a new server that has 40 cores that are each twice as fast as the current server, so the new site will be able to process these bans 20 times faster than the old server.

For now, website performance appears to be nominal, and the additional computing power is being brought online as a precaution if the number of IP addresses with these failed login attempts increases. Eventually, no additional computing power will be needed because so much of the Internet will already have been banned that there will be fewer bans to process no matter how many requests there are.

Even though there have been somewhere near a billion invalid password attempts, not a single one has been successful in stealing any money, so there has been no impact, other than someone burning a lot of cash to try the passwords.

[Banished_Privateer]

Are these attacks an attempt to compromise users' accounts with bruteforce/dictionary methods?

[Steve Sokolowski]

Are these attacks an attempt to compromise users' accounts with bruteforce/dictionary methods?

I don't know. Whatever is going on, the IP addresses aren't checking Google's "I'm not a robot" box, since they can't, so all of the addresses are banned after the first try. They're just wasting effort because they can't steal any money when all the requests can't solve the captchas.

There are billions of packets that are simply dropped because they just waste bandwidth not solving the captchas. We easily have enough CPU power now to deal with 20 times this many requests - and that assumes they are all new IP addresses, which they obviously don't have unlimited numbers of. As time goes on, more and more of the packets simply get dropped because a greater proportion of IP addresses are on the lists, so no further CPU power is needed to determine if the captchas are correct.

[spauk]

I'm not an expert on these kinds of things, but if all the IP addresses getting banned are just randomized spoofed addresses, what would happen if it was spoofed to look like the IP of a regular miner and that gets the miner banned? That would probably be unlikely but it sounds like you're expecting to ban possibly millions of addresses, is it even possible for someone to have access to that many real addresses? What happens to tor and VPN connections found trying to break passwords, can an IP address even be found or does captcha not work through that?

[TheSBG]

My home ip is banned... i can still access on mobile

[Steve Sokolowski]

My home ip is banned... i can still access on mobile

What is your username? Please log into your forums account from that home IP address to prove you have access to it, and I'll search for the address in the list.

[Steve Sokolowski]

I'm not an expert on these kinds of things, but if all the IP addresses getting banned are just randomized spoofed addresses, what would happen if it was spoofed to look like the IP of a regular miner and that gets the miner banned? That would probably be unlikely but it sounds like you're expecting to ban possibly millions of addresses, is it even possible for someone to have access to that many real addresses? What happens to tor and VPN connections found trying to break passwords, can an IP address even be found or does captcha not work through that?

There haven't been any confirmed support tickets of banned IP addresses. The bans don't require invalid password attempts; they require multiple invalid captcha attempts. Humans are very unlikely to fail the captchas multiple times in a row.

There are botnets that have millions of addresses, so I don't think that having a lot of addresses would be surprising. The criminals who run the botnets charge for their use, which is why I stated that the criminals are wasting a lot of money.

These bans can't be spoofed connections because HTTP requires the establishment of a connection. That means that the server sends a packet back to the IP address, and then that IP address has to respond to request the data. If these were false IP addresses from random packets, the IP address would never respond. These are either people knowingly participating in criminal activity, or who have been tricked into installing software that is being used for criminal activity.

Another reason random packets is unlikely is because the default Linux kernel will not send packets upstream if a downstream machine sends packets to it with an IP address out of the range it knows it has access to. Even if there are some bad routers that have disabled this feature somewhere, the largest ISPs will filter out junk packets upstream from them. I ran into this issue in 2016 when there was a lot of traffic coming to our system and I wanted to set up a VPN filter, but could not without adding additional latency.

[Banished_Privateer]

Humans are very unlikely to fail the captchas multiple times in a row.

Take my post with a grain of salt ;)