Forged Alliance Forever

Re: Server move - discussion thread

2020-01-17T01:09:02+02:00

We had to fix some ipv6 problems and reverse dns which cost us a little time, but overall the downtime was below 3 hours so we made it in time almost.

Please note: The missing flags is not related to the server migration. The Geolite database service changed and doesn't work with the current lobby server version. There will be a fix in the next days.

Statistics: Posted by Brutus5000 — 17 Jan 2020, 01:09

Re: Server move - discussion thread

2020-01-17T00:34:42+02:00

From what I can tell it's taking a bit longer, I don't think the server will be back by midnight.

Statistics: Posted by Wesmania — 17 Jan 2020, 00:34

Re: Server move - discussion thread

2020-01-16T23:55:26+02:00

Will the server be available at all tonight? Thanks for your work on it btw.

Statistics: Posted by Ferball — 16 Jan 2020, 23:55

Re: Server move - discussion thread

2020-01-14T23:34:03+02:00

The new scheduled date for the server migration is Thursday (16.01.) with a planned start of 21:00 CET +/- 30 minutes.

Statistics: Posted by Brutus5000 — 14 Jan 2020, 23:34

Re: Server move - discussion thread

2020-01-13T19:35:36+02:00

Organisational issues being my incompetence. I apologize go everyone and hope we can get the move executed swiftly.

Statistics: Posted by Sheeo — 13 Jan 2020, 19:35

Re: Server move - discussion thread

2020-01-13T10:16:54+02:00

Most of you have noticed that we had no downtime yesterday. Due to organisational issues we were temporarily unable to change the DNS. We have not planned when we will do the next try. We'll see.

Statistics: Posted by Brutus5000 — 13 Jan 2020, 10:16

Re: Server move - discussion thread

2020-01-11T12:38:47+02:00

For those who didn't read the news: It's supposed to happen tommorow.

Statistics: Posted by Brutus5000 — 11 Jan 2020, 12:38

Re: Server move - discussion thread

2020-01-08T21:40:19+02:00

Short update: The server is being installed right now. I hope we will do the dns-handover within this week. Unfortunately there was some delay on provisioning the server so we lost a few days and my holidays are over now.

Statistics: Posted by Brutus5000 — 08 Jan 2020, 21:40

Re: Server move - discussion thread

2019-12-21T00:39:55+02:00

Well... i guess the AX51-NVMe product from Hetzner is better for our application and it has ECC RAM and 2x1TB SSD, saves alot of problems. The difference is 24 EUR/month. Also ECC is kind of a must on servers.
In case of i can contribute myself with 50EUR for 2020

Statistics: Posted by sirbarbax — 21 Dec 2019, 00:39

Re: Server move - discussion thread

2019-12-21T00:30:19+02:00

A small tip. In case of database major version difference between new and old server, upgrade the old server to the same version as the new one. It can safe you a lot of headaches .
depends if you rsync the database, which is fastest to do

And I might be able to assist with advise or help, since it is my regular job facilitating migrations at a hosting company.

Statistics: Posted by misthafalls — 21 Dec 2019, 00:30

Re: Server move - discussion thread

2019-12-21T00:12:59+02:00

since i have 2 root @ hetzner which are let's say unused atm how about you use one of them?

specs are

CPUIntel Core i7-4770

HDD2x SSD SATA 240 GB

RAM4x RAM 8192 MB DDR3

second one is from 2010 with 2 750gb hdd

you can use it for testing / or until you get the new one

message me if you are interested

you can reinstall whatever os you see fit except windows ofc dont wanna pay for liscences xD

maybe outsource the vault?

Statistics: Posted by Syr3L — 21 Dec 2019, 00:12

Re: Server move - discussion thread

2019-12-19T22:26:51+02:00

I have to correct myself. It is indeed possible to change the server config if you click on the configurator...
But if we add another HDD it's already +12€ for a single non-raid hdd.
(And +6€ for ECC by the way)

Statistics: Posted by Brutus5000 — 19 Dec 2019, 22:26

Re: Server move - discussion thread

2019-12-19T21:46:36+02:00

Morax wrote:
Brutus wrote:Unfortunately there is one drawback. As you have seen the disk size shrank from 2 TB to 0,5 TB. Looking at our servers today we will need slightly more than 0,5TB. The biggest part of that is our replay vault and part of that seems to be duplicated, so some additional investigations required there, maybe we can reduce that. In worst case we would drop parts of the oldest replays which are most probably broken anyway. Another part of the solution is adding a new compression to the replay format, which would reduce replay size by ~20% (more information here).

How much more is a month to get 1TB raid config SSDs?

Rather spend time coding a compression algorithm, I would be fine donating some money dedicated to make up for the cost so we reduce the risk to 0%.

What OS?

Statistics: Posted by Little Miss Murder — 19 Dec 2019, 21:46

Re: Server move - discussion thread

2019-12-19T21:35:35+02:00

The one I have looked out is a rented, dedicated root server, but it's not possible to add additional hardware [edit: I'm wrong, this is possible]. Also it is non-ECC memory as it is consumer hardware, but so is the present server. The SSDs are NVMe. See the full details here: https://www.hetzner.de/dedicated-rootserver/ax41-nvme

I know it's not an enterprise server, but that's not what we are looking for. We can survive a service outage (the community doesn't have server admin adresses, so they can't pitchfork us )and rather go for a better price.
If it weren't for the money I'd go full cloud hosted solution in kubernetes environment, but for 45€ we wouldn't even get the managed sql db.

Regarding the file system: I have absolutely no experience with this. I know the advantages of ZFS and BTRFS but I don't have the skills to set this up or fix it if something breaks. I would have gone for regular Ubuntu 18.04 LTS with whatever it comes in the default installation (probably ext4).

I've seen your PM and I will invite you to our Zulip chat. We can discuss if you want to help on this topic.

Statistics: Posted by Brutus5000 — 19 Dec 2019, 21:35

Re: Server move - discussion thread

2019-12-19T21:06:28+02:00

I do have extensive experience (10 years give or take) with zfs (on solaris, nexenta and openindiana) use it as a corporate san solution
And consider https://zfsonlinux.org/ to be quite stable since release 0.7.8 (use it as a backup solution)
Protects vs bit rot, snapshots are a backup/restore/test heaven and the cache is MRU (Most Recent Used, not very exiting) and MFU (Most Frequent Used, this is the good stuff) so it will speed up harddisk access a lot (you can even add a ssd as secondary cache (l2arc) but for speed I would recommend: add more memory, zfs will use it all)
The memory of the new server is ECC right?
Are the ssd's sas or sata? (sata means be cautious with endurance and sudden failure, sas does to have more reliable prediction failure)
Considered a ssd/hardisk mix?
Is a owned server? (Can add our own hardware?)
If you want to go the sata ssd route, my advise: use 3 different brand ssd's with zfs in a 3 way mirror to prevent outage and data loss
( I tried a sata zfs server and while I monitored the smart endurance properties failures did occour after a few years...)
I would be happy to donate a 1tb sata ssd

Statistics: Posted by Mielus — 19 Dec 2019, 21:06