Server Infrastructure

[TP2k]

Hi,

While this sub-forum mainly focuses on code contributions, I would like to offer some sysadmin/infrastructure contribution.

After the recent blips of downtime, I've been thinking about how the server infrastructure might be improved. The basic premise is to split the various component so that if you have an issue with one, the others are not affected. This also has other benefits should FAF explode in popularity (that's what we all want right? )
Of course, I've had to make some assumptions about the current setup, so I'm happy to be corrected.....

Use an external IRC network
It would appear the chat currently runs on a single IRC server on the same box as the main FAF service. Assuming it's just a vanilla IRC server (and doesn't have any server-side mods to interact with the FAF service), you could consider moving the chat service to an existing IRC network. This has the following advantages:

• Someone else is maintaining a large, hopefully stable, network that we can make use for free.
  
• It frees up valuable resources on the FAF dedicated server.
  
• During a FAF outage, chat should still be available.

I have some opinions on which networks are better than others and how you might make FAF flexible enough to move between IRC networks easily if you wanted to peruse this.

Move the website to another server
It would appear the website runs on the same server as the main FAF service. If you moved this to another server you get some benifits:

• Any DoS against the website would not affect the main FAF service
  
• During a FAF outage, the website would still be available
  
• Free up more resources on the main dedi for the FAF service.

If the website just requires PHP & MySql, I would happily host it for free.


I am also interested in various deployment mechanisms which could be useful for rapidly deploying the FAF service to a new server in the case of a major outage. But a) I have no idea how the current server is set-up b) Small-baby steps

I would be interested to hear if Pilot thinks any of these are worthwhile.

[Ze_PilOt]

mptions about the current setup, so I'm happy to be corrected.....

Use an external IRC network

We need to have all nickname available for sure, that's not an option.

Move the website to another server

Not possible, some part of the website use some datas of the lobby (leader board, map vault, replay vault, and even the download link for the lobby). syncing the DB all the time will not really help freeing up ressources.

Also, the web components doesn't take many resources, and during downtime, we have facebook and twitter to communicate about the issues.

[ZombieTwiglet]

Having the website server split from the FAF server is a no brainer, having everything running on one server is a recipe for disaster and terrible for any sudden spikes in popularity. Like with the RAID issue the other week it took everything down, hell the website and logins went down overnight the other day for a good few hours.

How is that "not possible"? You can query a database on one server from another server with ease no need to sync databases, this isn't a performance issue it's a redundancy issue hell even just an accessibility issue, if the login server is down you can say on the website and vicea versa. Sure FAF isn't massive at the moment but relying on one server that performs multiple roles is a stupid idea (just ask any sysadmin), hell you can spinup a VPS for the website so you have full control over the system rather than 'traditional' hosting and you can ensure it's got all the resources it needs there is zero need code wise for any database to be on the same physical box as the website hell we couldn't have sites like Amazon or Facebook if that was the case.

I'm not hating at all and I'm glad we have people like you keeping the community alive but when everything goes inaccessable at the same time that community has nowhere to go.

[Ze_PilOt]

It will stays that way.

You say that if one server goes down, I can query the db of another one. That's true, but you forgot that the software need to run too.
So yes, the DB needs to be in sync all the time, it's not just a switch in queries (if the main server goes down, the server service too).

The performance hit of the forum/front website is so low that moving it away will virtually change nothing.
In fact, I don't know why you think that the server is overloaded and need to be split. It's really not We are not google or facebook
(and, by the way, when the main server of each continent is going down for battle.net, you have 0 redundancy and you have to wait for them to put it back up. And we are talking of a company that earn millions each month).

The server is almost never down (uptime this month is close to 100%, there was a little downtime of 2 hours last week), so finding a better way of doing this is a loss of time for a project this size.

[CoolCold]

Looks like this topic is a bit outdated, but let's try anyway.
I've seen message about traffic proxying and extra bills for project, so , i can try to host such proxy on my dedicated server @Germany (Hetzner.de). i've not discussed this with hoster yet, but if this is interesting in general for the project, I will.
I'm running debian linux with openvz virtualization/isolation, so can even create VE (VM) for the project goals. I've 10Tb of traffic included, guess this gonna be enough ;)