export hotstats to html

Everything about mods can be found here.

Moderator: Morax

Re: export hotstats to html

Postby SeraphimLeftNut » 18 Jan 2013, 06:08

The key missing statistics are mass and energy being spent as a function of time. Incomes are nice, but usage is much more interesting.
Looks like the time step can't be less than 0.5 seconds based on your suggestion, but that is already pretty good.

There also seems to be an upper limit on log size that the program is able to handle, which excludes long setons games.

The x hr, x min, x sec label also complicates data import from the source file. In places where you have 1 hour 30 seconds, it would be nicer if it said 1 hour, 0 minutes, 30 seconds.
no ui lag: https://www.youtube.com/watch?v=3MdcVdL2kIY
I think this is going to be fun
User avatar
SeraphimLeftNut
Contributor
 
Posts: 975
Joined: 10 Jul 2012, 19:46
Has liked: 3 times
Been liked: 219 times
FAF User Name: TA4Life

Re: export hotstats to html

Postby peanot » 18 Jan 2013, 11:56

Thanks, I will see if I can get an interactive prototype running soon.

Oh also, can someone tell me where I can find the generated log file? I seem to be missing something obvious here.
peanot
 
Posts: 5
Joined: 17 Sep 2012, 18:38
Has liked: 0 time
Been liked: 0 time
FAF User Name: peanot

Re: export hotstats to html

Postby fffmmm » 18 Jan 2013, 15:47

SeraphimLeftNut wrote:The key missing statistics are mass and energy being spent as a function of time. Incomes are nice, but usage is much more interesting.
Looks like the time step can't be less than 0.5 seconds based on your suggestion, but that is already pretty good.

There also seems to be an upper limit on log size that the program is able to handle, which excludes long setons games.

The x hr, x min, x sec label also complicates data import from the source file. In places where you have 1 hour 30 seconds, it would be nicer if it said 1 hour, 0 minutes, 30 seconds.

I updated it a little:
-you can now export the data to a CSV-File (Not sure if the data in the CSV is organized that well... Let me know if that needs to be adjusted)
-it now acquires a stats snapshot every second instead of every 10 seconds
-added a chart for the mass spending rate and the energy spending rate (overcharges look great on that one)
-changed the way it prints the time so that it always prints it in the format Xh Xm Xs



peanot wrote:Thanks, I will see if I can get an interactive prototype running soon.

Oh also, can someone tell me where I can find the generated log file? I seem to be missing something obvious here.

You have to enable logging in the lobby (Options->settings->save game logs). The logs can be found in the following directory: C:\ProgramData\FAForever\logs
User avatar
fffmmm
Contributor
 
Posts: 8
Joined: 16 Jul 2012, 18:54
Has liked: 0 time
Been liked: 0 time
FAF User Name: fffmmm

Re: export hotstats to html

Postby rootbeer23 » 18 Jan 2013, 16:59

i would prefer to see mass reclamation and mass income from extractors and fabricators seperately.

in dump.lua, you should write a single line of LOG() for each tick that is just a collection of values, instead of
dumping the whole table, including the keys (of the lua associative array).
i.e. dump a csv with the values
s["resources"]["massin"]["total"], s["resources"]["massin"]["rate"], ...

finally i suggest to parse the replay.log with python and thus make the parser integral part of falobby and then add a new tab to falobby that shows the resulting html.
rootbeer23
Supreme Commander
 
Posts: 1001
Joined: 18 May 2012, 15:38
Has liked: 0 time
Been liked: 31 times
FAF User Name: root2342

Re: export hotstats to html

Postby rootbeer23 » 18 Jan 2013, 17:41

or better yet, save the values in a flat array while accumulating the scores instead of copying the scoreData into the historical array. that whould cut down on RAM waste by several factors.
rootbeer23
Supreme Commander
 
Posts: 1001
Joined: 18 May 2012, 15:38
Has liked: 0 time
Been liked: 31 times
FAF User Name: root2342

Re: export hotstats to html

Postby fffmmm » 18 Jan 2013, 17:51

rootbeer23 wrote:i would prefer to see mass reclamation and mass income from extractors and fabricators seperately.

in dump.lua, you should write a single line of LOG() for each tick that is just a collection of values, instead of
dumping the whole table, including the keys (of the lua associative array).
i.e. dump a csv with the values
s["resources"]["massin"]["total"], s["resources"]["massin"]["rate"], ...

finally i suggest to parse the replay.log with python and thus make the parser integral part of falobby and then add a new tab to falobby that shows the resulting html.

JSON objects are easily parsed using python, thus I think it's a good choice (Even easier to parse than CSV).
Writing the snapshot regularly to the log would cause a lot of IO ops and make it more difficult to parse I think.

I tried to find a way to get the reclaim for everyone, but sadly I only found a function to get it for ones own economy, but not for everyones eco.
User avatar
fffmmm
Contributor
 
Posts: 8
Joined: 16 Jul 2012, 18:54
Has liked: 0 time
Been liked: 0 time
FAF User Name: fffmmm

Re: export hotstats to html

Postby SeraphimLeftNut » 18 Jan 2013, 17:58

For me the ascii format, is the format of freedom.
This is awesome
no ui lag: https://www.youtube.com/watch?v=3MdcVdL2kIY
I think this is going to be fun
User avatar
SeraphimLeftNut
Contributor
 
Posts: 975
Joined: 10 Jul 2012, 19:46
Has liked: 3 times
Been liked: 219 times
FAF User Name: TA4Life

Re: export hotstats to html

Postby rootbeer23 » 18 Jan 2013, 18:10

fffmmm wrote:JSON objects are easily parsed using python, thus I think it's a good choice (Even easier to parse than CSV).


String[] strArr=line.split(",");
if you only have numbers in the array which is the case. otherwise there are full blown csv parsers for any programming language.

fffmmm wrote:Writing the snapshot regularly to the log would cause a lot of IO ops and make it more difficult to parse I think.


you can still write the data after the game if you want.
anyway, if you want to do it the wrong way, i dont wanna argue.

fffmmm wrote:I tried to find a way to get the reclaim for everyone, but sadly I only found a function to get it for ones own economy, but not for everyones eco.


then maybe its not possible, i was just assuming since gazui can do it, hotstats should be able to also.
rootbeer23
Supreme Commander
 
Posts: 1001
Joined: 18 May 2012, 15:38
Has liked: 0 time
Been liked: 31 times
FAF User Name: root2342

Re: export hotstats to html

Postby SeraphimLeftNut » 18 Jan 2013, 23:15

Mass use vs time data has a lot of oscillations in it (see first image attached), that correspond to factories going on and off as units come out of them. The frequency of this oscillation is dependent on the unit being produced, build capacity associated with that factory and available resources.
Second image is FFT of this data(sliding a 3 minute window, one second at a time). Red color is zero, blue color is intense. Vertical axis is the detected frequencies going from low to high(bottom to top).
As the game progresses you see more and more high frequencies, which are associated with air production and increased build capacity on factories being used.

The oscillations in power are also present.

Unfortunately I only have access to data from a single random game right now. I expect these plots to be very telling in setons games, especially for the air players. (allies that leech off allies for power will have the oscillations of those they leech from imprinted on their eco)

Also the plots would be much better if the sampling rate wasn't 1 second. This hides a lot of things, by forcing the use of a wide 3 minute window, which smears out dynamic events.
Attachments
IRI_mass_use_vs_time.png
IRI_mass_use_vs_time.png (6.02 KiB) Viewed 2879 times
IRI_mass_FFT_2D.png
IRI_mass_FFT_2D.png (48.78 KiB) Viewed 2879 times
no ui lag: https://www.youtube.com/watch?v=3MdcVdL2kIY
I think this is going to be fun
User avatar
SeraphimLeftNut
Contributor
 
Posts: 975
Joined: 10 Jul 2012, 19:46
Has liked: 3 times
Been liked: 219 times
FAF User Name: TA4Life

Re: export hotstats to html

Postby SeraphimLeftNut » 18 Jan 2013, 23:17

Analogous power data for the same game, same player.

Looks like someone was power stalling. (that removes the oscillations)
Attachments
IRI_power_use_vs_time.png
IRI_power_use_vs_time.png (6.04 KiB) Viewed 2878 times
IRI_power_FFT_2D.png
IRI_power_FFT_2D.png (45.56 KiB) Viewed 2878 times
no ui lag: https://www.youtube.com/watch?v=3MdcVdL2kIY
I think this is going to be fun
User avatar
SeraphimLeftNut
Contributor
 
Posts: 975
Joined: 10 Jul 2012, 19:46
Has liked: 3 times
Been liked: 219 times
FAF User Name: TA4Life

PreviousNext

Return to Mods & Tools

Who is online

Users browsing this forum: No registered users and 1 guest