Forged Alliance Forever

Re: How should FFAs be rated?

2016-08-28T19:00:06+02:00

It doesn't take long to gain eg. 500 rating points, even with an account that has hundreds of games played before, if your skill level really is higher. Sure it is easier with new accounts to gain rating fast, but if you can't get past 700 I don't think it is because of the rating system.

Statistics: Posted by Viba — 28 Aug 2016, 19:00

Re: How should FFAs be rated?

2016-08-28T18:07:52+02:00

I've played over 500 team game's and its impossible to get past 700 because of a bad start.

If you have a skilled friend to help you from the beginning you end up over a 1000 after a few game's easily.

Just wish ratings were based on last 50 to 100 games or something like that after you get to understand what to do.

Statistics: Posted by Gr1nder — 28 Aug 2016, 18:07

Re: How should FFAs be rated?

2016-08-26T04:44:27+02:00

I think one of the most substantial disagreements about scoring the FFA game is because we do not have a clearly defined objective. As I note there are two different objectives, and how you score the game is dependent on what you see the objective as being. For example:

1. The objective of an FFA is to survive.
2. The objective of an FFA is to kill the other players.

Now at a first glance these may appear to be the same thing. But actually they are not.

Personally, I would think that the greatest utility in scoring an FFA will happen when you take all measurable outcomes into account and meter these against the likelihood of those outcomes occurring. But this is dependant on valuing all those outcomes - and considering whether or not the low-kill FFA victor is as deserving as one with a lot of kills. Personally, I think that the more active FFA player is more deserving of score. So I think the inclusion of kills is relevant for score... But this is a personal opinion.

Now the basic approach might be to treat the FFA game as a series of coinciding 1v1's - but because the level of uncertainty will increase with an increasing number of players. Basically because the additional interactions create more uncertainty. The way this would work in your example would be:

(1) Scoring kills - Player B (1400) kills Player D (1600). So we treat this as though it were a 1v1 between a 1400 vs. 1600 player and score this outcome accordingly. Though, because it is an FFA, we modify the uncertainty calculation. Which is not just a function of Sigma for players B and D, but, is also a function of the number of players within the game. So because Player B (1400) killed player D (1600) this is an unexpected result, and therefore will result in a large transfer of points from player D to player B. Though, because of the higher uncertainty, the transfer of points will not be as large as might be expected had this been a 1v1 match.

(2) Scoring a win/survival - according to your analogy Player A killed Players B and C, winning the FFA. So addition to the score for kills on player B and player C, player A should also be awarded score for winning the game. Which should be measured against the likelihood of winning that game. Something which is a function of both the aggregate skill level, and the number of players in the game.... For instance if we assume that all players have the same skill, the chances of winning a 4-way FFA is 25%, and one should expect that as one's score is higher or lower than the aggregate value, one's chances of winning will be higher or lower than 25% accordingly.

I can't give you exact formulas, but, this is how I would approach the problem.

Statistics: Posted by Hawkei — 26 Aug 2016, 04:44

Re: How should FFAs be rated?

2016-08-26T01:13:49+02:00

everywhere116 wrote:
What happens when you set them all to 1 instead of one 0 and eleven 1's?

None of the means change, but several of the new ratings have different sigma values.

Statistics: Posted by Softly — 26 Aug 2016, 01:13

Re: How should FFAs be rated?

2016-08-26T01:06:11+02:00

Softly wrote:
Maybe someone can explain why identically ranked players get different numbers of points for the same score?

Code:
#testing trueskill for FFAs import trueskill from trueskill import Rating trueskill.setup(mu=1500, sigma=500, beta=240, tau=10, draw_probability=0.10) r = Rating(mu=1500,sigma=60) res = trueskill.rate([[r],[r],[r],[r],[r],[r],[r],[r],[r],[r],[r],[r]],ranks=[0,1,1,1,1,1,1,1,1,1,1,1]) for rating in res: print(rating) """ Results: (trueskill.Rating(mu=1512.769, sigma=59.718),) (trueskill.Rating(mu=1498.466, sigma=59.174),) (trueskill.Rating(mu=1498.578, sigma=59.162),) (trueskill.Rating(mu=1498.676, sigma=59.153),) (trueskill.Rating(mu=1498.761, sigma=59.147),) (trueskill.Rating(mu=1498.834, sigma=59.144),) (trueskill.Rating(mu=1498.895, sigma=59.144),) (trueskill.Rating(mu=1498.945, sigma=59.146),) (trueskill.Rating(mu=1498.985, sigma=59.151),) (trueskill.Rating(mu=1499.014, sigma=59.158),) (trueskill.Rating(mu=1499.033, sigma=59.169),) (trueskill.Rating(mu=1499.043, sigma=59.184),) """

What happens when you set them all to 1 instead of one 0 and eleven 1's?

Statistics: Posted by everywhere116 — 26 Aug 2016, 01:06

Re: How should FFAs be rated?

2016-08-26T00:29:09+02:00

Easy: global rating has too many factors and I hardly consider someone's outcome in a FFA to be useful, quantitative assessment of skill in other games.

Statistics: Posted by Morax — 26 Aug 2016, 00:29

Re: How should FFAs be rated?

2016-08-26T00:20:24+02:00

Maybe someone can explain why identically ranked players get different numbers of points for the same score?

Code:: #testing trueskill for FFAs import trueskill from trueskill import Rating trueskill.setup(mu=1500, sigma=500, beta=240, tau=10, draw_probability=0.10) r = Rating(mu=1500,sigma=60) res = trueskill.rate([[r],[r],[r],[r],[r],[r],[r],[r],[r],[r],[r],[r]],ranks=[0,1,1,1,1,1,1,1,1,1,1,1]) for rating in res: print(rating) """ Results: (trueskill.Rating(mu=1512.769, sigma=59.718),) (trueskill.Rating(mu=1498.466, sigma=59.174),) (trueskill.Rating(mu=1498.578, sigma=59.162),) (trueskill.Rating(mu=1498.676, sigma=59.153),) (trueskill.Rating(mu=1498.761, sigma=59.147),) (trueskill.Rating(mu=1498.834, sigma=59.144),) (trueskill.Rating(mu=1498.895, sigma=59.144),) (trueskill.Rating(mu=1498.945, sigma=59.146),) (trueskill.Rating(mu=1498.985, sigma=59.151),) (trueskill.Rating(mu=1499.014, sigma=59.158),) (trueskill.Rating(mu=1499.033, sigma=59.169),) (trueskill.Rating(mu=1499.043, sigma=59.184),) """

Statistics: Posted by Softly — 26 Aug 2016, 00:20

Re: How should FFAs be rated?

2016-08-25T23:58:51+02:00

That's probably because you're on your own and not on a team. I noticed that global 1v1s affect your rank a lot more than teamgames do as well.

Statistics: Posted by everywhere116 — 25 Aug 2016, 23:58

Re: How should FFAs be rated?

2016-08-25T23:15:00+02:00

well in ffa you tend to go to the highest threat, so lets say you are lowest rated player in the game, just sit there being useless. and someone else dies. great now you got rating for free. in my case i died second and gained 20 rating. was a long time ago so dont know the details but yeah

another example was when speed2 lost 40 rating cos i built a czar and killed everyone at once. his rating was decided completely by me. i could have killed in any order but i had a nice outcome on his rating. it was me violet, speed and someone else like sheeo. i also got like 30-40 rating

in any case its strange since you get 10x the rating change from an ffa than you get from a teamgame.

Statistics: Posted by Exotic_Retard — 25 Aug 2016, 23:15

Re: How should FFAs be rated?

2016-08-25T22:12:03+02:00

Morax wrote:
I agree with Philip... FFA is kind of a weird game mode that adds too much variation to the global rating of players.

What do you mean by variation? If you have some specific cases you want to discuss, let me know and I will post the exact numbers.

Personally, I think the best argument is coming up with results like this:

3 way FFA, two 1.6k (A and B) players and a 1.4k player (C) (as ever, with sigma=60 for all players).

If player C ctrl+ks 10 seconds into the game, they gain points (1.9 points to be exact).

This effect increases with a rating gap, suppose we have A and B on 2k and C on 1k. In that case player C gains 29 points (Nice return for a 10 second ctrl+k).

Statistics: Posted by Softly — 25 Aug 2016, 22:12

Re: How should FFAs be rated?

2016-08-25T21:55:26+02:00

I agree with Philip... FFA is kind of a weird game mode that adds too much variation to the global rating of players.

Statistics: Posted by Morax — 25 Aug 2016, 21:55

Re: How should FFAs be rated?

2016-08-25T21:49:29+02:00

Why does FFA have to be rated at all?

Spoiler: show

Statistics: Posted by PhilipJFry — 25 Aug 2016, 21:49

Re: How should FFAs be rated?

2016-08-25T21:38:52+02:00

For clarity, when I say "scoring from games" I mean how the game reports the score to the server. Last I checked (years ago) it was 1 point per ACU kill, -1 per death, and 5 for surviving.

This had some odd consequences, for example ctrl+k in a 1v1 to kill the opponent would lead you to win (your score is 0, the opponent -1).

In teamgames the team score is aggregated, so everyone on the same team gets the same treatment regardless of who got the kills.

Statistics: Posted by Softly — 25 Aug 2016, 21:38

Re: How should FFAs be rated?

2016-08-25T21:19:44+02:00

Softly wrote:
One caveat: I am assuming that scoring from games works the same way now as it did ~4 years ago (probably true though).

It was changed a bit about 3 weeks ago: https://github.com/FAForever/server/com ... 2c16ce7eca
Here's the thread with the info on why it was changed: viewtopic.php?f=45&t=11698&hilit=axle

As for your post, I had no idea kills decided where you placed in an FFA, that sounds bizarre.

Statistics: Posted by JaggedAppliance — 25 Aug 2016, 21:19

Re: How should FFAs be rated?

2016-08-25T21:12:04+02:00

Also the code (in python3.5) for people who want to try this at home:

Code:: #testing trueskill for FFAs import trueskill from trueskill import Rating trueskill.setup(mu=1500, sigma=500, beta=240, tau=10, draw_probability=0.10) r1 = Rating(mu=2000,sigma=60) r2 = Rating(mu=1500,sigma=60) print(trueskill.rate([[r1],[r2],[r1],[r2]],ranks=[0,1,2,2]))

trueskill settings taken from: https://github.com/FAForever/server/blo ... fig.py#L16

Statistics: Posted by Softly — 25 Aug 2016, 21:12