Forged Alliance Forever

Re: Rating Decay, and why we need it desperately.

2015-12-17T04:21:19+02:00

Aulex wrote:
I'm fairly certain noCaps wants to create a new rating system catered for FAF after we spent a night looking at glicko/True skill, glicko2 and elo.

I don't think trueskill is flawed. The implementation we've used for most of our existence is most likely flawed, however.

Statistics: Posted by Sheeo — 17 Dec 2015, 04:21

Re: Rating Decay, and why we need it desperately.

2015-12-17T03:18:14+02:00

I'm fairly certain noCaps wants to create a new rating system catered for FAF after we spent a night looking at glicko/True skill, glicko2 and elo.

Statistics: Posted by Aulex — 17 Dec 2015, 03:18

Re: Rating Decay, and why we need it desperately.

2015-12-17T03:12:47+02:00

Morax wrote:
Maybe in order to avoid the "noobs getting slaughtered" scenario we use a well-thought and intelligent rating assignment to well-known players.

E.G. give Zock his 2k+ and etc...

I'm sure we can all reach and concur upon a rating for folks in the top 200?

Also, this would remove the pain of players who have 100s-to-1000s of games having to achieve those numbers again. Everyone else can get a "second chance."

This is nonsensical. The rating system only works if it gets to apply the ratings for all players equally. Without the rating algorithm assigning ratings itself you're left with numbers that have arbitrary meanings.

Statistics: Posted by Sheeo — 17 Dec 2015, 03:12

Re: Rating Decay, and why we need it desperately.

2015-12-16T20:16:33+02:00

I like the idea of rating decay, it works well in LoL

But on a serious note, I played a spurt of ladder for about 3-4 weeks where my rating climbed from 1200 all the way to around 1550. And now a few months later I am trying to play at that same rating and just getting my ass kicked. It would be nice to remove the getting my ass kicked part of it from my rating going down to where it should be until I get back into the swing of 1v1 again.

Something worth discussing is what would be an appropriate length for seasons. A month feels too short, I think somewhere from 2-4 would be best with prizes for the top 10 at the end of each season.

Statistics: Posted by briang — 16 Dec 2015, 20:16

Re: Rating Decay, and why we need it desperately.

2015-12-16T14:33:46+02:00

Or not hard reset but just a soft one, like that you avoid a 0 playing against 2k and making him leave the game because he's crushed every game.

Because anyway a 2k is not going to play like a 100, and a 1k is not going to play like a 100 too.

Statistics: Posted by Masamune — 16 Dec 2015, 14:33

Re: Rating Decay, and why we need it desperately.

2015-11-02T16:57:27+02:00

Maybe in order to avoid the "noobs getting slaughtered" scenario we use a well-thought and intelligent rating assignment to well-known players.

E.G. give Zock his 2k+ and etc...

I'm sure we can all reach and concur upon a rating for folks in the top 200?

Also, this would remove the pain of players who have 100s-to-1000s of games having to achieve those numbers again. Everyone else can get a "second chance."

Statistics: Posted by Morax — 02 Nov 2015, 16:57

Re: Rating Decay, and why we need it desperately.

2015-11-01T02:47:54+02:00

Blodir wrote:
I remember in the past you were talking about reworking rating entirely and recalculating it. Is this still something you want to do, and is it something you'd like to do before implementing any rating decay? I'm a bit concerned with how long it will take, however if there is going to be a rating rework then it should happen after teamgame matchmaker is complete so we can finally have accurate teamgame ratings.

While I do think league system with seasons etc. is a good idea (in fact it was an idea that I often suggested and endorsed in LoL years ago before they talked about implementing it), but again I'm concerned with how long implementing these changes will take. I think it's a bit overly ambitious considering even developers like Blizzard have had problems balancing such a system (look at HotS), it's much simpler leaving rating unabstractified and a lot of people prefer it that way too.

Yes, and sure -- but we can do both. You can recompute ratings retroactively while including deviation decay as well.

Statistics: Posted by Sheeo — 01 Nov 2015, 02:47

Re: Rating Decay, and why we need it desperately.

2015-10-30T11:31:23+02:00

The first problem I see with this is that for people who have jobs, family, life, playing regularly is not so easy. It would suck to see points disappear in thin air only because you didn't play for 3 weeks. Talking about a motivation drain, OP. For the top 50 ladder try hards this obviously isn't an issue, for the other 99% of FAF it is.

Also, when you have been away for a while, the last thing you need is an incredible punch in the rating when you are getting back in the groove. Simple example, in summer time most people are away for a good period of time. Give people some time to adjust, even if they play like a 1300 or 1400 instead of a 1600 right away. Do you care that much about supposed balance? Some times I wish rating had never been introduced. GPGnet used to have good games without it.

To me it seems that the issue raised by OP is hardly relevant for anyone except the top 50 of ladder. Does everyone have to bleed to enable a handful of players to reach a deserved spot in ladder ranking? Is it really that important?

Why not hide inactive players or smurfs first?

Statistics: Posted by Col_Walter_Kurtz — 30 Oct 2015, 11:31

Re: Rating Decay, and why we need it desperately.

2015-10-30T11:28:42+02:00

Sheeo wrote:
Some technical insight:

No reason to modify trueskill mean, simply apply a weekly buff of deviation by an absolute amount (+8 or 9 is probably fine), to a cap of say 250.

Instead of not applying the deviation increase for the initial three weeks, you might as well apply it always. Playing games regularly will compensate for the increase.

In practice this means the conservative estimate (Your displayed rating) will go down by ~100 points per month if you don't play, up to a maximum of 750 over time (3*250). A deviation of 250 will decrease by at least 30 points when playing vs an evenly matched opponent, that is a jump in 90 diplayed rating for playing a single game.

It allows your mean to jump either up or down to where it belongs (Trueskill will handle this), and gives some more flexibility to the rating evolution.

To implement this non retroactively is very straight forward and not really a large amount of work. To implement it retroactively (That is, in between games played before this point in time) is a lot more work, since you need to go back and adjust all rating assignments for games played.

P.S. inactive players are hidden on the leaderboards already.

I remember in the past you were talking about reworking rating entirely and recalculating it. Is this still something you want to do, and is it something you'd like to do before implementing any rating decay? I'm a bit concerned with how long it will take, however if there is going to be a rating rework then it should happen after teamgame matchmaker is complete so we can finally have accurate teamgame ratings.

While I do think league system with seasons etc. is a good idea (in fact it was an idea that I often suggested and endorsed in LoL years ago before they talked about implementing it), but again I'm concerned with how long implementing these changes will take. I think it's a bit overly ambitious considering even developers like Blizzard have had problems balancing such a system (look at HotS), it's much simpler leaving rating unabstractified and a lot of people prefer it that way too.

Statistics: Posted by Blodir — 30 Oct 2015, 11:28

Re: Rating Decay, and why we need it desperately.

2015-10-30T10:16:01+02:00

Some technical insight:

No reason to modify trueskill mean, simply apply a weekly buff of deviation by an absolute amount (+8 or 9 is probably fine), to a cap of say 250.

Instead of not applying the deviation increase for the initial three weeks, you might as well apply it always. Playing games regularly will compensate for the increase.

In practice this means the conservative estimate (Your displayed rating) will go down by ~100 points per month if you don't play, up to a maximum of 750 over time (3*250). A deviation of 250 will decrease by at least 30 points when playing vs an evenly matched opponent, that is a jump in 90 diplayed rating for playing a single game.

It allows your mean to jump either up or down to where it belongs (Trueskill will handle this), and gives some more flexibility to the rating evolution.

To implement this non retroactively is very straight forward and not really a large amount of work. To implement it retroactively (That is, in between games played before this point in time) is a lot more work, since you need to go back and adjust all rating assignments for games played.

P.S. inactive players are hidden on the leaderboards already.

Statistics: Posted by Sheeo — 30 Oct 2015, 10:16

Re: Rating Decay, and why we need it desperately.

2015-10-27T18:32:03+02:00

Well-said, Zock.

Might I suggest one more thing that I thought of last night:

1) Keep a ladder "career" stats that show ratings from the beginning of time (no reset, ever).

2) As stated, implement a seasonal ladder that is reset for use to determine more current ratings and tournament brackets.

This would take away some of the pain of a "reset" for those who have worked hard to take the top-tier spots and they won't be forgotten.

Statistics: Posted by Morax — 27 Oct 2015, 18:32

Re: Rating Decay, and why we need it desperately.

2015-10-27T15:52:05+02:00

speed2 wrote:
I wonder who will decide this, player councilor or whole council, if we ever get to the point of making decisions.

We already discussed this issues and decided measures to improve the situation a while ago. The ladder will get a rework, including seasons and a reset each season. (and more)

This is not to be confused with the trueskill ratings though. We use them as ladder substitute right now, because our "ladder" has its known issues. With a real ladder in place, we can hopefully use trueskill only for its intended purpose that is to generate even matches for players and won't have to make changes to it that will make it worse at this purpose in favour of a better representation of competition.

This might turn out to be wrong, but we should wait to see how good the new ladder system works before deciding if further changes are required.

When will the new system be done? As it requires coding work, and the current focus of the few people that dedicate their free time to work on faf is on improving the proxy situation, it will take a bit longer than everyone would like, but the issues with the ladder are not forgotten or ignored.

Everyone is welcome to speed it up by contributing to the coding of it.

Hope this was helpful and brought some light into what's going on.

Statistics: Posted by Zock — 27 Oct 2015, 15:52

Re: Rating Decay, and why we need it desperately.

2015-10-27T10:27:08+02:00

Resetting the ladder would have the effect that noobs will be kicked by good people again for a long time until rating is set more or less properly. Cant see the advantage of a rating reset in general. The ratings are according to player skill no matter if the person did have a long time without games. What a lot of people dont see when shouting on others is that everybody can have a bad day and mess up completely maybe even several games in a row. Sometimes people try new tactics and fail because not everybody has 5 hours time per day for sandboxing before playing 5 hours games testing this new strategy. It might be true that it was not good to seperate ratings 1v1 and global without reset but does it really matter? If those people play and are really bad they are rating cows thats ok because no other player is harmed. Good 1v1 players are good in teamgames as well after some training games.

I think the point of the global rating is to let it go and develop for a long time period and if you reset this rating you spoil it for a long time until everybody is back to his position. Ladder reset i recommed only if reimplementing real seasons. A permanent ladder ongoing is boring in my opinion. Better to have seasons back and make season mappool a bit bigger but for a longer time.

I am ok with anything ladder reset or not. Just wanted to mention that now i am in teamgames with more or less proper people and if you reset this t3 bombs will fall on t1 noobs for quite a time just to reach more or less the same ranking as before for the majority. If you really wanna change something simply change the rating calculation and deviation or how it is called.

Edit: Maybe increase the deviation over inactive time so the first games after inactivity or playing not much in general will have higher effect and reach rating position faster.

Statistics: Posted by Iszh — 27 Oct 2015, 10:27

Re: Rating Decay, and why we need it desperately.

2015-10-27T09:34:45+02:00

I didn't read this whole topic but I agree that ratings need resets every now and then!

Statistics: Posted by lextoc — 27 Oct 2015, 09:34

Re: Rating Decay, and why we need it desperately.

2015-10-27T08:51:10+02:00

I agree that rating decay makes sense.....

Having seasons where the rating is reset every few months would exacerbate the player mismatch problem discussed in the ladder matchmaker thread. The system would be completely unable to determine game quality for a while once you wipe all that data.

Statistics: Posted by Eric_Lesch — 27 Oct 2015, 08:51