Forged Alliance Forever

Re: [split from visionik AMA] Database Discussion

2014-09-29T22:53:02+02:00

I doubt that a nosql solution would be faster then mysql running ad hoc queries on 'min rating', 'player', 'map' order by date with limit xxx on one machine only. It all depends on the criteria for the search though; is it static data(write and never update), will the search method change in the future, how far can it be denormalized without forcing large codechanges/data migration, is simple backup important (put the replays in the filesystem).

The only good way to find out is to replicate/create good test data, run a few tests on a "small" machine or you limit it with docker/virtual machine, double the size of the data and see if its still running well. This is again only valid if the system is not doing anything else though so ... I´ve had my share of optimizing DB systems (Postgresql & some MSSQL) and we are running a cassandra setup with spark at my current work, but faf is not really running on a standard setup, you would normally have far more ram, faster disks and dedicated machines (DB on its own machine with fast network).

Statistics: Posted by maov — 29 Sep 2014, 22:53

Re: AMA: Ask Visionik "the owner" anything

2014-09-28T11:15:52+02:00

Vee wrote:
Yet it's possible to push all replay data to a for example a document-based database (in addition to the MySQL database) and it would be way easier and faster to query from that because of the lack of relations (you would push user data with the replay data with which you need to filter).

OMG...

What?

Statistics: Posted by Aurion — 28 Sep 2014, 11:15

Re: AMA: Ask Visionik "the owner" anything

2014-09-28T11:15:45+02:00

Vee wrote:
OMG...

Can you specify it more?
Why it is bad?
Why it is good?

Statistics: Posted by Dragonfire — 28 Sep 2014, 11:15

Re: AMA: Ask Visionik "the owner" anything

2014-09-28T10:54:43+02:00

Also mySql is not the best solution for such a big system ...
Never worked with it: googleBigTable, Hadoop

was not clear:
A relational database is not good for everything in faf, e.g. replays.

Yet it's possible to push all replay data to a for example a document-based database (in addition to the MySQL database) and it would be way easier and faster to query from that because of the lack of relations (you would push user data with the replay data with which you need to filter).

OMG...

Statistics: Posted by Vee — 28 Sep 2014, 10:54

Re: AMA: Ask Visionik "the owner" anything

2014-09-27T15:42:30+02:00

Ze_PilOt wrote:
partytime wrote:
Dragonfire wrote:A relational database is not good for everything in faf, e.g. replays.

I don't see why not. I heard there are only a million replays.

Its more than That.

Also you are not querying the replay table.

You are querying a table that has a row for every player in the game.

Yet it's possible to push all replay data to a for example a document-based database (in addition to the MySQL database) and it would be way easier and faster to query from that because of the lack of relations (you would push user data with the replay data with which you need to filter). This requires quite some storage, because it's not being stored efficiently of course and databases like that like lots of memory too.

Omnipotent wrote:
Organise a 'Kickstarter' for new map making tool... v60 scmaps, EnvCubes and custom decals and texture support.

Just throwing that one out there lol!

That would be awesome, but do we actually have the knowledge in this community to make something like that?

Statistics: Posted by Aurion — 27 Sep 2014, 15:42

Re: AMA: Ask Visionik "the owner" anything

2014-09-27T14:17:00+02:00

partytime wrote:
Dragonfire wrote:A relational database is not good for everything in faf, e.g. replays.

I don't see why not. I heard there are only a million replays.

Its more than That.

Also you are not querying the replay table.

You are querying a table that has a row for every player in the game.

Statistics: Posted by Ze_PilOt — 27 Sep 2014, 14:17

Re: AMA: Ask Visionik "the owner" anything

2014-09-27T14:11:26+02:00

Dragonfire wrote:
A relational database is not good for everything in faf, e.g. replays.

I don't see why not. I heard there are only a million replays.

Statistics: Posted by nine2 — 27 Sep 2014, 14:11

Re: AMA: Ask Visionik "the owner" anything

2014-09-27T12:26:09+02:00

ax0lotl wrote:
Dragonfire wrote:A relational database should be fine for FAF. Hadoop is for a whole different magnitude of data.

I was not clear:
A relational database is not good for everything in faf, e.g. replays.

Statistics: Posted by Dragonfire — 27 Sep 2014, 12:26

[split from visionik AMA] Database Discussion

2014-09-27T12:22:14+02:00

Dragonfire wrote:
Also mySql is not the best solution for such a big system ...
Never worked with it: googleBigTable, Hadoop

A relational database should be fine for FAF. Hadoop is for a whole different magnitude of data.

Statistics: Posted by ax0lotl — 27 Sep 2014, 12:22