The Last Outpost

General Discussion => Announcements => Topic started by: Veritas on March 16, 2014, 05:57:40 PM

Title: Downtime Report - March 16th
Post by: Veritas on March 16, 2014, 05:57:40 PM
Hi all,

As most of you know, the site was down from about 11:50 AM to 12:50 PM today, March 16th, 2014. This was the result of a DDoS attack against our server utilizing what's known as a UDP flood - an enormous amount of worthless incoming connections. As a result of this attack, our IP address was "nullrouted" - removed from operation - by our host for an hour. After the nullroute was in place, the attack stopped.

To mitigate future attacks, we've moved our server into a protected address range provided by our host. This protection carries some extra monthly cost, but should easily handle any additional attacks (up to roughly 100 times the strength of the attack we experienced this morning). Due to the switch, our old IP address is no longer accessible, and the server is now updating to the new address (DNS propagation should be complete within the next 12 hours or so).

I know that one of the biggest questions members might want answered is "who would do this to us?" Frankly, that's something I'm wondering myself. However, there isn't really a way to know, and guessing about it doesn't really help us in any way - regardless of who is doing this, our response is the same. We will continue to take measures to ensure TLO remains free and accessible for everyone.

If anyone has any questions, please don't hesitate to contact me.

Veritas
Title: Re: Downtime Report - March 16th
Post by: Lipton on March 17, 2014, 07:05:41 AM
So this was the website that got the attack, correct? Not the two RP servers?

As to the attack itself, all I have to say is this: Some people will do any kind of mischief to get their kicks.
Title: Re: Downtime Report - March 16th
Post by: Veritas on March 17, 2014, 09:40:08 AM
Yes. Only the website's server (which also hosts Teamspeak) was affected.
Title: Re: Downtime Report - March 16th
Post by: Paenitet on March 18, 2014, 11:08:54 AM
Is it really worth spending more money to stop something like this from happening again or can this attack actually be very dangerous?
Title: Re: Downtime Report - March 16th
Post by: Veritas on March 18, 2014, 11:42:01 AM
Depends on your definition of "dangerous." User data and so forth are not at risk, but if such attacks were allowed to persist, eventually TLO might be offline more than it's online - plus, hosts do not want to host people who are the target of frequent, unmitigated DDoSes. We have compromised the quality of our hosting services to save money before, and ended up paying dearly for it. I don't think it's reasonable to take risks on this kind of thing, especially since we have a significant financial surplus and can sustain the cost.
Title: Re: Downtime Report - March 16th
Post by: Serris on March 18, 2014, 08:02:49 PM
Was the attack specifically targeted at TLO? I can't think of a reason why. Maybe it was directed at another site and we just happened to get caught in the crossfire? I've had that happen with other web hosts.
Title: Re: Downtime Report - March 16th
Post by: Veritas on March 18, 2014, 09:06:50 PM
It was directed right at our IP address and only at our IP address, based on what our host has told me. There's every indication this was targeted at us, but again, speculation is not gonna change anything. Regardless of the who and why, we need to be protected.
Title: Re: Downtime Report - March 16th
Post by: Veritas on March 22, 2014, 08:40:36 PM
Update: We have fixed an issue which was causing some users to be blocked by the DDoS filtering for innocuous usage. If this happens to you, please contact me with the instructions here (http://www.last-outpost.net/forum/index.php/topic,6659.new.html#new).
Title: Re: Downtime Report - March 16th
Post by: Veritas on April 08, 2014, 10:49:06 AM
Another update on this:

While I think I have cleared up all instances of TLO users losing access to the site, there is still a strong possibility that other users are being blocked and I'm not hearing about it, or that the filtering is preventing new users from signing up.

To abrogate these difficulties, I will be moving TLO's webserver to a separate VPS located in BuyVM's Las Vegas datacenter, which has similarly robust filtering without the issues we're currently experiencing. The only downside to this will be that the LV datacenter does not support TeamSpeak (or any UDP service), meaning TeamSpeak will be moved to a different server to compensate (there will be no cost for this; I am absorbing it).

The move will happen later this week, before this weekend. I will keep everyone advised as we proceed.

Thank you all for your patience, and as always, don't hesitate to contact me with questions.
Title: Re: Downtime Report - March 16th
Post by: Veritas on April 09, 2014, 06:16:24 PM
Update: We've moved to the new server. Service should resume as normal, and nobody should be getting blocked.

DNS can take up to a day to propagate, but If anyone has any issues accessing the site, please contact me.