Maybe its time to move the Ladder starting skill down.

Talk about general things concerning Forged Alliance Forever.

Moderators: FtXCommando, Ze Dogfather

Re: Maybe its time to move the Ladder starting skill down.

Postby Sheeo » 31 Jan 2017, 14:52

Downlord wrote:For anyone interested, our wiki describes how TrueSkill works: http://wiki.faforever.com/index.php?tit ... kill_works

The relevant part is this: "By default, you have 1500 in mean, and 500 in deviation. 1500 is the average level."

So the system assumes that new players have a skill (NOT rating) of 1500 which, as we probably all agree, is very unlikely (btw. the average player's average skill at this point is 1348). This leads to new people facing ~1200 rated players.


That's not quite what an initial rating of 1500, 500 means, see the PDF (probability density function) of the normal distribution for N(1500, 500^2): https://www.wolframalpha.com/input/?i=N(1500,+500%5E2)

It means that the system assigns a probability of your skill being 1500: roughly 0.08%.
Support FAF on patreon: https://www.patreon.com/faf?ty=h

Peek at our continued development on github: https://github.com/FAForever
Sheeo
Councillor - Administrative
 
Posts: 1038
Joined: 17 Dec 2013, 18:57
Has liked: 109 times
Been liked: 233 times
FAF User Name: Sheeo

Re: Maybe its time to move the Ladder starting skill down.

Postby Sheeo » 31 Jan 2017, 14:54

RealityCheck wrote:Sheeo, I assumed changing the starting skill is as simple as changing two or three numbers. Is it actually much more complex?


Yes, that's right. It's very simple to change initial parameters -- but that doesn't mean it's correct or helpful to do so. It just means we introduce false information into trueskill and worsen the relation between assigned ratings and actual skill.
Support FAF on patreon: https://www.patreon.com/faf?ty=h

Peek at our continued development on github: https://github.com/FAForever
Sheeo
Councillor - Administrative
 
Posts: 1038
Joined: 17 Dec 2013, 18:57
Has liked: 109 times
Been liked: 233 times
FAF User Name: Sheeo

Re: Maybe its time to move the Ladder starting skill down.

Postby Sheeo » 31 Jan 2017, 14:55

sasin wrote:
Sheeo wrote:This is a fundamental problem with treating TrueSkill as a ladder system. Tokyto is drafting a new ladder system which doesn't use TrueSkill directly for bracketing, which would solve this problem.

The proposed solution of just lowering the initial rating would just defer the systems ability to estimate the skill level of newer players until much later -- which doesn't really solve the problem of mid-range players being afraid of getting matched against smurfs.


Why not lower the mean but keep the deviation high? Mean of 800, deviation of 500, or some such.


It may be a "quick fix" to lower the initial mean, so long as we keep the deviation high, yes. But I wouldn't trust this to not induce inflation over time.
Support FAF on patreon: https://www.patreon.com/faf?ty=h

Peek at our continued development on github: https://github.com/FAForever
Sheeo
Councillor - Administrative
 
Posts: 1038
Joined: 17 Dec 2013, 18:57
Has liked: 109 times
Been liked: 233 times
FAF User Name: Sheeo

Re: Maybe its time to move the Ladder starting skill down.

Postby biass » 31 Jan 2017, 15:01

How quick?
Map thread: https://bit.ly/2PBsa5H

Petricpwnz wrote:biass on his campaign to cleanse and remake every single map of FAF because he is an untolerating reincarnation of mapping hitler
User avatar
biass
Contributor
 
Posts: 2239
Joined: 03 Dec 2015, 07:54
Has liked: 598 times
Been liked: 662 times
FAF User Name: biass

Re: Maybe its time to move the Ladder starting skill down.

Postby sasin » 03 Feb 2017, 23:50

Sheeo wrote:
sasin wrote:
Sheeo wrote:This is a fundamental problem with treating TrueSkill as a ladder system. Tokyto is drafting a new ladder system which doesn't use TrueSkill directly for bracketing, which would solve this problem.

The proposed solution of just lowering the initial rating would just defer the systems ability to estimate the skill level of newer players until much later -- which doesn't really solve the problem of mid-range players being afraid of getting matched against smurfs.


Why not lower the mean but keep the deviation high? Mean of 800, deviation of 500, or some such.


It may be a "quick fix" to lower the initial mean, so long as we keep the deviation high, yes. But I wouldn't trust this to not induce inflation over time.


I'm genuinely curious, how do you think it'd induce inflation over time? For players who already have a rating playing against a new player, it would cause their rating to increase by less, right? I'm imagining the net effect wouldn't be very big, but the effect on already playing players would be ever so slightly lower ratings, then? Even if that were the case, if you're playing against enough new players that your rating is enhanced by beating up on them, then I think it's probably more accurate for your rating to go down a little.

For the new players, it would cause them to start with a lower rating, which I think is our goal here. That way, they don't get matched against good players.

Also, if you're worried about new players then affecting the more experienced player's ratings too much, you could make it so anyone who has played less than 10 games has no effect on their opponent's ratings. Not sure how hard that would be to code, but should address everyone's problems, right?
sasin
Priest
 
Posts: 368
Joined: 11 Feb 2013, 04:09
Has liked: 26 times
Been liked: 43 times
FAF User Name: sasin

Re: Maybe its time to move the Ladder starting skill down.

Postby Exotic_Retard » 04 Feb 2017, 00:34

i would expect the opposite effect - rating deflation - since whenever a new player joins theres this effect where they have a juicy 1500 rating and proceed to lose it all - injecting essentially free rating into the system. if we give them less to lose then we can expect deflation not inflation? also couple the fact that a bunch then leave the game and therefore never get it back off other noobs or their old rating farmers.

in any case the system will be affected by this change. im kinda tempted to say it would be for the better though.
User avatar
Exotic_Retard
Contributor
 
Posts: 1470
Joined: 21 Mar 2013, 22:51
Has liked: 557 times
Been liked: 626 times
FAF User Name: Exotic_Retard

Re: Maybe its time to move the Ladder starting skill down.

Postby sasin » 04 Feb 2017, 00:54

Exotic_Retard wrote:i would expect the opposite effect - rating deflation - since whenever a new player joins theres this effect where they have a juicy 1500 rating and proceed to lose it all - injecting essentially free rating into the system. if we give them less to lose then we can expect deflation not inflation? also couple the fact that a bunch then leave the game and therefore never get it back off other noobs or their old rating farmers.

in any case the system will be affected by this change. im kinda tempted to say it would be for the better though.


Yeah, as per my post I'd expect deflation, not inflation, although I'm curious as to hear if sheeo is seeing something we're not.

I'm guessing the deflation would be very minimal though. Like you, I think it'd only be for the better, because anyone profiting off of beating up on the new players to inflate their rating has an undeservedly high rating.
sasin
Priest
 
Posts: 368
Joined: 11 Feb 2013, 04:09
Has liked: 26 times
Been liked: 43 times
FAF User Name: sasin

Re: Maybe its time to move the Ladder starting skill down.

Postby biass » 04 Feb 2017, 08:02

It would be cool to see a vote on a quick fix if it can at least get some people playing and won't cause negative consequence, instead of waiting years for new systems
Map thread: https://bit.ly/2PBsa5H

Petricpwnz wrote:biass on his campaign to cleanse and remake every single map of FAF because he is an untolerating reincarnation of mapping hitler
User avatar
biass
Contributor
 
Posts: 2239
Joined: 03 Dec 2015, 07:54
Has liked: 598 times
Been liked: 662 times
FAF User Name: biass

Re: Maybe its time to move the Ladder starting skill down.

Postby speed2 » 04 Feb 2017, 15:33

Exotic_Retard wrote:i would expect the opposite effect - rating deflation - since whenever a new player joins theres this effect where they have a juicy 1500 rating and proceed to lose it all - injecting essentially free rating into the system. if we give them less to lose then we can expect deflation not inflation? also couple the fact that a bunch then leave the game and therefore never get it back off other noobs or their old rating farmers.

in any case the system will be affected by this change. im kinda tempted to say it would be for the better though.

Sheeo wrote:
Downlord wrote:For anyone interested, our wiki describes how TrueSkill works: http://wiki.faforever.com/index.php?tit ... kill_works

The relevant part is this: "By default, you have 1500 in mean, and 500 in deviation. 1500 is the average level."

So the system assumes that new players have a skill (NOT rating) of 1500 which, as we probably all agree, is very unlikely (btw. the average player's average skill at this point is 1348). This leads to new people facing ~1200 rated players.


That's not quite what an initial rating of 1500, 500 means, see the PDF (probability density function) of the normal distribution for N(1500, 500^2): https://www.wolframalpha.com/input/?i=N(1500,+500%5E2)

It means that the system assigns a probability of your skill being 1500: roughly 0.08%.

Retard there is no juicy 1500 rating to lose
User avatar
speed2
Contributor
 
Posts: 3189
Joined: 05 Jan 2013, 15:11
Has liked: 636 times
Been liked: 1119 times
FAF User Name: speed2

Re: Maybe its time to move the Ladder starting skill down.

Postby Exotic_Retard » 04 Feb 2017, 15:59

true, however that 0.08% is the highest change of your rating being there. i would think that a noob starting at supcom would have a higher chance being 300 rated than 1500 rated, which is not what this PDF shows.

so my point still stands, to get to a more accurate representation of your rating you need to lose 5-10 games first, in the majority of cases, since what the rating system assumes is not the distribution of actual new players.

what we could do is take all players under 50 games, and see their average rating, then use that as the starting mean + deviation. Im very sure thats its not 1500 +-1500.
User avatar
Exotic_Retard
Contributor
 
Posts: 1470
Joined: 21 Mar 2013, 22:51
Has liked: 557 times
Been liked: 626 times
FAF User Name: Exotic_Retard

PreviousNext

Return to General Discussions

Who is online

Users browsing this forum: No registered users and 1 guest