The Spooky Database

[everywhere116]

Since the real unit database is down we have to rely on the Spooky database in order to get our unit info. This has a couple problems, one of which is being out of date witht he current patches, which is fixable pretty easily, and another which I'll get into below. Before we begin, I just want to say that I love the Spooky database for FAF and it's design and how it works, it's much superior to the default FAF version in terms of design. However, it isn't always reliable and here is why.



Above we see a comparison among the four T2 Mobile Flaks in the game. They all have a similar DPS, except for the Sky Boxer. Why? Well, the Skyboxer projectile has a much lower damage than the other three, and to compensate for that the Sky Boxer launches two projectiles per shot instead of one like all the others, however the fire cycle shown here shows that it only shoots one projectile per half-second, when we know it shoots two. Because the database thinks the Sky Boxer shoots less projectiles than it does it calculates its DPS incorrectly.



Here is the code for how it calculates all of these. As highlighted in the red box, the code calls for the variable MuzzleSalvoSize from the unit blueprint and uses it as the number of projectiles the unit fires in a firing cycle. Let's look at that variable in the Sky Boxer's blueprint:



Huh, that can't be right, the MuzzleSalvoSize variable equals one. And this was extracted from the nx2 file from the FAF client, so we know it's correct. But the Sky Boxer shoots twice per cycle. What's going on?

Well, as it turns out, it's not the variable we actually want...



This is what we want, ProjectilePerOnFire, which for the Sky Boxer equals two, just as it should be. So the database's code for calculating the firing cycle doesn't pull the right variable. That seems odd.

It turns out Spooky made a slight mistake. The MuzzleSalvoSize variable does tell you how many shots a unit takes in a firing cycle, but only if the shots are taken in succession like with the Ravager, but the ProjectilePerOnFire variable tells you how many shots are taken at the same time. So units like the Summit Class and Pillar will have the number of projectiles the fire governed by the ProjectilePerOnFire variable (and thus will suffer from this glitch) and units like the Ravager will have the number of shots they take governed by the MuzzleSalvoSize variable, and thus will not. The Mongoose both suffers and doesn't suffer from this glitch, since it's plasma gattling cannon won't be affected by this but it's grenade launcher will (the database says it launches 4 shots per 6.7 seconds but it actually shoots 3 grenades per round, so it's DPS is underestimated by two-thirds)

Normally to fix this I would recommend to just multiply weapon.MuzzleSalvoSize by weapon.ProjectilesPerOnFire, but it isn't that simple, at least as far as I can tell. We're getting into territory that I am uncertain about, so I may be wrong, but I don't believe it actually reads the unit blueprint files, instead it reads from these two json files in the source code.



Both of these files look like this:



So basically dumps of all the blueprint data into one file. The second file is smaller than the first though and it looks like it has a lot of information stripped from it that is present in the first file but seems irrelevant, things like animation bones and audio cues and things like that. This leads me to believe that the database reads from the smaller second file for all of it's information.

Unfortunately one of the pieces of information stripped from the second file is ProjectilesPerOnFire. It doesn't appear anywhere. D'oh.

So that would need to be added back into the file....and unfortunately that's where my usefullness ends. I looked through the rest of the program and I found what looks like the files used to dump and strip the blueprint files of their data, but I can't find anywhere that says what data it stripped and which data it didn't. For what it's worth, here's where I believe it strips the data from the index.fat.json file:



And I'm not a javascript programmer (or any kind of programmer for that matter, unless you count extremely rudimentary QBasic) and I can't find anything in this part that says which parts are kept and which parts are stripped, and it references files I can't find anywhere else in the databases source code. Hopefully Spooky himself knows and can fix it.

Taken from my original post about this on Imgur, http://imgur.com/a/TJlCa I decided to transcribe it here. Also I'm pretty sure that this kind of thing can go on this forum, but if it doesn't I apologize.

[KeyBlue]

Nice explanation and research.

Some of my thoughts and findings:

• The 'real' unitdatabase does work: http://direct.faforever.com/faf/unitsDB/
  or is there another one you are referring too?
  
  
• And i figured out how his script knows which parts need to be kept and which parts are stripped.
  His script literally looks at the .js and .html files and checks which values they use.
  So even if you add another value in your calculations or displays, it will extract it from the file and keep it in.
  He does it with the regex which is pretty difficult to figure out, since it's not obvious what its purpose is.
  See an example
  
  So basicly, you don't need to think about the filtering, because it'll extract what you need from your files.
  (you do need to specify the files though)
  But it's not 100% foolproof since he had to add some of them manually apparantly.
  
• The creator mentions this license http://www.wtfpl.net/
  Which means we can use it, i guess?
• So someone with knowledge regarding dps and other stats (balance team?) could fix the calculations, so it works properly.
  And then we can replace it for the current one, which is also pretty outdated.

[everywhere116]

Huh, your link works The one you got by selecting the "unit List" in the client did not, and still doesn't load properly now even though the link technically works.

As for the code stuff, wow, it's like a self referential code loop, he made the code say which variables he wanted and then told the datamining tool to search for the information he asked for in his own code. I'm impressed. Hopefully by running the same tool again on the current nx2 file with the updated code can fix both the out-of-date issue and the salvo bug.

As for fixing it in terms of math, the only thing we'd need to do is multiply the MuzzleSalvoSize variable by the ProjectilesPerOnFire variable in the code where the MuzzleSalvoSize variable currently is. The only issue is whether or not there are any units that have a weapon that don't have a ProjectilesPerOnFire variable for whatever reason, although I can't imagine why there would be. But not every unit has a MuzzleSalvoSize either and that's why he made it check to see if the weapon has a manual fire toggle as well before checking for MuzzleSalvoSize.

[KeyBlue]

Well apparantly there is already improvements waiting to be added.
And there has been talk about hosting it ourselves.

https://github.com/spooky/unitdb/pull/57

So i guess this just needs to get done and not have it sit there for a year before it actually happens.

[Sprouto]

The ProjectilesPerOnFire value was deprecated long ago and should do nothing. In fact, it's not present in most units at all.

The real reason the Sky Boxer has two projectiles is because it has 2 muzzle bones - which are coded to fire together. So, the real way to get the true damage value is to make sure that all the muzzles are being counted -- and if they are set to fire together or not - and then multiply by MuzzleSalvoSize.

[JoonasTo]

Funny you should pick the Summit as an example as it's one of the units that DOESN'T work properly...

Spoiler: show

Like mentioned here and somewhere else a little while ago, things with weapons that fire linked, or weapons with multiple barrels have problems. The UEF T2 point defense(Triad), the UEF T2 destroyer(Valiant), etc. etc.

EDIT: Got Ninja'ed

[everywhere116]

I said that the Summit DOES suffer from the glitch I am referring too, hence why it's DPS in the Spooky database is 1.3 of what it is in the main database.

As for the muzzle bones being the reason why the Sky Boxer and other units fire multiple projectiles and not the ProjectilesPerOnFire variable, is there a way to determine that systematically without adding a value like that in manually for every applicable unit? Also, how does the main database avoid this error?

[ABSTRACT]

i noticed a very similar problem in the standart unit db, when i wondered about the low dps of Aeon T3 Torpedo bomber.

Spoiler: show

http://direct.faforever.com/faf/unitsDB/unit.php?bp=XAA0306

it calculates 500 dmg for 4 projectiles: 4*500dmg = 2000 in total; 12,5 sec recharge -> 160 dps.
in fact 4 projectiles are 4 DOUBLE projectiles so its 4*2*500dmg = 4000 in total, and 320 dps.
it was very confusing.

[IceDreamer]

ProjectilePerOnFire is a completely nonfunctional variable. Do NOT use it.

[IceDreamer]

I said that the Summit DOES suffer from the glitch I am referring too, hence why it's DPS in the Spooky database is 1.3 of what it is in the main database.

As for the muzzle bones being the reason why the Sky Boxer and other units fire multiple projectiles and not the ProjectilesPerOnFire variable, is there a way to determine that systematically without adding a value like that in manually for every applicable unit? Also, how does the main database avoid this error?

if MuzzleSalvoSize is 1 AND RackFireTogether is true then it fires X projectiles where X is the number of racks. If RFT is false, it will fire 1 rack, then the next, on each RoF cycle. If MSS is > 1 then will fire each rack in sequence until it saturates the number of shots, each RoF cycle, and ignores RFT value.