Downtime Update and Q&A

[Veritas]

Final update from Veritas:
Hi all,

I have another update and I'm afraid it's not a good one. According to our host, there is no additional data they can recover from the crash. This means that any forum posts, members, or other content not present on this forum right now is not going to become available again.

I apologize sincerely once more for this incident. It's as much my fault as it was that of our host, since I was not as diligent as I could have been in ensuring backups were not only taken, but functional. You all have my personal assurance that in the future, this will be completely rectified. Rest assured that until the site is functioning again, all TLO members will have my full attention for any assistance I can render.

Senators, I will be posting a proposal for how we should proceed with rebuilding the site, including a breakdown of the costs involved, either tonight or tomorrow morning, including an urgent vote on the proposal (with multiple options). As soon as that vote passes I will make the necessary purchases and begin setting up the new server. At this time, the site and forum will be my priority, but my proposal will also cover suggestions for the Minecraft situation and outline a comprehensive solution for regular backups of the site going forward.

Again, if there are any questions, comments, suggestions or concerns, please don't hesitate to contact me.

Veritas

Original post continues below:

Hi everyone,

First, here's a copy of the statement which now appears on our homepage:

The Last Outpost - Status Update

Around 4PM EST yesterday (2 November 2012), The Last Outpost's webhost was subject to an attack on its administration system. The attacker attempted to delete data from several of the host's server nodes, including the nodes where TLO hosted our web, Minecraft, and Teamspeak servers.

While not all the data on our site was lost, and external backups do exist, it is possible at this time that up to a month of content from our forum and site have been lost, along with the majority of our Minecraft server data. Unfortunately, until we're informed by our host, we won't have any specifics on what exactly was recovered, and what we stand to lose.

We have setup a temporary forum for members and guests alike so that you can discuss the situation and continue to organize gaming events and RPs. Our RPG-X, Armada, Bridge Commander and Garry's Mod servers remain online, and our fleet in STO remains active. Please join us there, and don't hesitate to ask us if you have any questions or concerns.

The Last Outpost Senate and staff apologize sincerely for the inconvenience. We'll keep you updated through this page, the new forum, and our Facebook and Twitter feeds.

Please find below a few questions and answers about the status of the forum and site:

What happened to all of my posts after mid-September?
As you can see, the temporary forum only contains content (accounts, PMs, posts, etc) from late September. Around four-five weeks of information is currently unavailable to us; whether it still exists on our host, we're still not certain. What's on the forum now represents the most recent backup we saved locally, as opposed to on our host's central backup server, which is unavailable. While we hope to eventually recover this content, we don't know right now how quickly that will happen.

My account doesn't exist. What do I do?
If you signed up after September 19th, your account doesn't exist on this version of the forum. Please resignup with the same e-mail you used originally. If you were a TLO member or had other special positions, message Veritas, JohnF, or Jenson, and we will setup your account with the correct privileges (be sure to include the rank you held and your uniform color, if applicable). 

Are my account, e-mail and password safe?
At this time, we have no reason to believe any of our data was stolen - the attacker only attempted to delete data from our host's nodes, and wasn't after anything specific as far as we are aware. There's no reason to assume that your account has been compromised at this time, but if you're concerned, we recommend changing your password on this site and any other sites where you used the same one.

What about Minecraft, Teamspeak, and the other servers?
Most of TLO's other services were not affected. Services which continue to run normally include both RPG-X servers, Garry's Mod, Armada 2, Bridge Commander, and all of TLO's in-game fleets.

However, both our TeamSpeak and Minecraft servers were hosted with our webhosting provider. It appears that all data for our Minecraft and Teamspeak servers was destroyed, though, again, we don't know that with any certainty yet. This means that both of those servers might need to start anew, or at least be reverted to a much older interation. We'll provide more information on them as it becomes available.

Where does TLO go from here?
Over the next few days, we'll be trying to assess the damage and deciding what the best solution is going forward. We continue to hope that our host will recover the missing data from the past month. Regardless of whether that happens, we'll be looking into where we can continue to host our content going forward, whether that means staying with our current host or switching to a new one.

If you have any additional questions, comments or concerns, please don't hesitate to contact us. Again, we apologize for the inconvenience, and hope you'll stick with us while we rebuild.

Veritas & the technical team

[Grizz]

Are we just the unluckiest gaming community out there? If we don't get attacked at least once a month at every host we have, I get concerned.

[Veritas]

To be fair, we've been with this host since January without any issues like this before...

[Spock]

I trust this will not affect any scheduled events for this weekend, such as SRPs?

[TriggerAngry]

Well, lets hope that git who attacked us doesn't come back here. I have every faith in you Veritas, and I'm sure you'll get us back up and running.

[Veritas]

I trust this will not affect any scheduled events for this weekend, such as SRPs?

With only the exception of Asteroid Base (as far as I know), everything will run as normal.

Well, lets hope that git who attacked us doesn't come back here. I have every faith in you Veritas, and I'm sure you'll get us back up and running.

I appreciate that . To be clear, we specifically weren't attacked - only the company who hosts are servers. I don't want anyone to think TLO is being targeted here.

[Spock]

I trust this will not affect any scheduled events for this weekend, such as SRPs?

With only the exception of Asteroid Base (as far as I know), everything will run as normal.

Actually, AB is going on as normal again

[SFC3]

 AB is still on for Sunday. So don't be alarmed.

[Veritas]

For Senators and members alike, here's some more information on what we're considering: http://lastoutpost.x10.mx/forum/index.php?topic=4361.0

[Veritas]

Update: Just got an e-mail from our host, who states they've had a harder time recovering data than they anticipated. I will reach out to them and keep you all informed of their response. Thanks again for your patience.

In the meantime, members are invited to check out the thread regarding this issue in the Senate forums for a more complete description of what we're planning to do, and to can discuss it in the Governance Forum if you're so inclined.

[Apprentice]

Maybe it's an idea to redirect the last-outpost.net url to this forum instead ??

[Veritas]

...it is being redirected. Try it in your browser.

[SFC3]

Redirect the forums as well?

[Veritas]

Redirect the forums as well?

Hopefully we'll have our server back tonight - if not, yes, I will do that.

Here's another quick update: We have recovered a partial copy of the data from our Minecraft server. While we don't yet know what exactly was recovered, it appears to only contain data up through early August of this year. Our host is still working on our website, and we'll inform you once we have more information on that.

[Veritas]

Hi all,

I have another update and I'm afraid it's not a good one. According to our host, there is no additional data they can recover from the crash. This means that any forum posts, members, or other content not present on this forum right now is not going to become available again.

I apologize sincerely once more for this incident. It's as much my fault as it was that of our host, since I was not as diligent as I could have been in ensuring backups were not only taken, but functional. You all have my personal assurance that in the future, this will be completely rectified. Rest assured that until the site is functioning again, all TLO members will have my full attention for any assistance I can render.

Senators, I will be posting a proposal for how we should proceed with rebuilding the site, including a breakdown of the costs involved, either tonight or tomorrow morning, including an urgent vote on the proposal (with multiple options). As soon as that vote passes I will make the necessary purchases and begin setting up the new server. At this time, the site and forum will be my priority, but my proposal will also cover suggestions for the Minecraft situation and outline a comprehensive solution for regular backups of the site going forward.

Again, if there are any questions, comments, suggestions or concerns, please don't hesitate to contact me.

Veritas