Server move - discussion thread

[Brutus5000]

This is the ultimate discussion thread around the upcoming server move as announced in the news.

If you have any questions, this is the right place to ask.

[---]

"So that means an additional 50% of logical cores plus the technical improvements of the last 8 years! Without having exact comparisons it should basically double CPU performance. Moving from HDD to SSD will also have a huge boost on database performance. The additional RAM is a nice goodie, but not really needed for now."

What will the actual effect be (especially of the CPU upgrade)? Replay vault will be faster with searches?
Game itself won't be affected, correct?

[Morax]

Unfortunately there is one drawback. As you have seen the disk size shrank from 2 TB to 0,5 TB. Looking at our servers today we will need slightly more than 0,5TB. The biggest part of that is our replay vault and part of that seems to be duplicated, so some additional investigations required there, maybe we can reduce that. In worst case we would drop parts of the oldest replays which are most probably broken anyway. Another part of the solution is adding a new compression to the replay format, which would reduce replay size by ~20% (more information here).

How much more is a month to get 1TB raid config SSDs?

Rather spend time coding a compression algorithm, I would be fine donating some money dedicated to make up for the cost so we reduce the risk to 0%.

[Brutus5000]

What will the actual effect be (especially of the CPU upgrade)? Replay vault will be faster with searches?
Game itself won't be affected, correct?

Every query to the database (replay, map, mod, achievements) should be faster.
But the more important thing is that we fix the problems we usually get in the evenings where people can't login because the server is too busy.

[Mielus]

I do have extensive experience (10 years give or take) with zfs (on solaris, nexenta and openindiana) use it as a corporate san solution
And consider https://zfsonlinux.org/ to be quite stable since release 0.7.8 (use it as a backup solution)
Protects vs bit rot, snapshots are a backup/restore/test heaven and the cache is MRU (Most Recent Used, not very exiting) and MFU (Most Frequent Used, this is the good stuff) so it will speed up harddisk access a lot (you can even add a ssd as secondary cache (l2arc) but for speed I would recommend: add more memory, zfs will use it all)
The memory of the new server is ECC right?
Are the ssd's sas or sata? (sata means be cautious with endurance and sudden failure, sas does to have more reliable prediction failure)
Considered a ssd/hardisk mix?
Is a owned server? (Can add our own hardware?)
If you want to go the sata ssd route, my advise: use 3 different brand ssd's with zfs in a 3 way mirror to prevent outage and data loss
( I tried a sata zfs server and while I monitored the smart endurance properties failures did occour after a few years...)
I would be happy to donate a 1tb sata ssd

[Brutus5000]

The one I have looked out is a rented, dedicated root server, but it's not possible to add additional hardware [edit: I'm wrong, this is possible]. Also it is non-ECC memory as it is consumer hardware, but so is the present server. The SSDs are NVMe. See the full details here: https://www.hetzner.de/dedicated-rootserver/ax41-nvme

I know it's not an enterprise server, but that's not what we are looking for. We can survive a service outage (the community doesn't have server admin adresses, so they can't pitchfork us )and rather go for a better price.
If it weren't for the money I'd go full cloud hosted solution in kubernetes environment, but for 45€ we wouldn't even get the managed sql db.

Regarding the file system: I have absolutely no experience with this. I know the advantages of ZFS and BTRFS but I don't have the skills to set this up or fix it if something breaks. I would have gone for regular Ubuntu 18.04 LTS with whatever it comes in the default installation (probably ext4).

I've seen your PM and I will invite you to our Zulip chat. We can discuss if you want to help on this topic.

[Little Miss Murder]

Unfortunately there is one drawback. As you have seen the disk size shrank from 2 TB to 0,5 TB. Looking at our servers today we will need slightly more than 0,5TB. The biggest part of that is our replay vault and part of that seems to be duplicated, so some additional investigations required there, maybe we can reduce that. In worst case we would drop parts of the oldest replays which are most probably broken anyway. Another part of the solution is adding a new compression to the replay format, which would reduce replay size by ~20% (more information here).

How much more is a month to get 1TB raid config SSDs?

Rather spend time coding a compression algorithm, I would be fine donating some money dedicated to make up for the cost so we reduce the risk to 0%.

What OS?

[Brutus5000]

I have to correct myself. It is indeed possible to change the server config if you click on the configurator...
But if we add another HDD it's already +12€ for a single non-raid hdd.
(And +6€ for ECC by the way)

[Syr3L]

since i have 2 root @ hetzner which are let's say unused atm how about you use one of them?

specs are

CPUIntel Core i7-4770

HDD2x SSD SATA 240 GB

RAM4x RAM 8192 MB DDR3

second one is from 2010 with 2 750gb hdd


you can use it for testing / or until you get the new one

message me if you are interested

you can reinstall whatever os you see fit except windows ofc dont wanna pay for liscences xD

maybe outsource the vault?

[misthafalls]

A small tip. In case of database major version difference between new and old server, upgrade the old server to the same version as the new one. It can safe you a lot of headaches .
depends if you rsync the database, which is fastest to do

And I might be able to assist with advise or help, since it is my regular job facilitating migrations at a hosting company.