Shuffle iT Forum

Dominion => Feature Requests => Topic started by: Sharur on 08 October 2017, 06:15:01 PM

Title: Increase spread of player ratings
Post by: Sharur on 08 October 2017, 06:15:01 PM
This post describes the rating system used by Dominion Online: http://forum.shuffleit.nl/index.php?topic=1679.msg5891#msg5891

The part I am following up on is:

QuoteIt's quite possible that 50 + 7.5 * (μ-2φ) is not very good at mapping everyone to the 0-100 range. If too many people end up at 0, or if someone goes above 100, we'll change the 50 or the 7.5 later on, but that won't impact the actual rating system under the hood.

I'm honestly not sure why someone going over 100 would be a problem, but in any case, the data is in, and that is not a risk. The top player has a rating of... 66. I think I may have seen someone get as high as 67 or 68, but definitely not 70.

While ultimately it doesn't really matter (since the numbers that actually matter are the Glicko-2 numbers), this means that too many players are bunched together. Once you've played a good number of matches, your rating moves very slowly, even if you're winning all or almost all of your games*.

There are a couple of easy fixes to this. Personally, I would remove the transformation, and just show μ-2φ directly, and carry it out to enough significant digits (probably two decimal places) that people can see a noticeable difference between them and players not too far from them on the leaderboard**. If you think that people don't like decimals, just multiply by 100, or however much you need to get all of your significant digits in front of the decimal.

Another easy fix is to leave the transformation as is, and just display more significant digits. Just adding one decimal would probably be enough to provide the differentiation and the sense that your rating actually responds to individual games.

Or, the not as easy solution - actually update the transformation. For that I assume you'd just have to play with the constants to see what you'd get. I do find it odd that for players who have played enough games, i.e. their φ (volatility) approaches 0, their rating does not center on 50 for average play, unless an average μ (rating) is 0, but I don't think it is. But perhaps there will always be enough players who haven't played a ton of games that centering the ratings on 50 requires accounting for significantly negative μ-2φ values.

*This is exacerbated by the latency of leaderboard updates, which is another issue/topic. It would help if I knew when the leaderboard updated, but as it stands the combination of slow moving rating and updates which take place seemingly no more frequently than once/day make it very hard to tell when my rating has updated.

**The inability to see those immediately around you on the leaderboard is another issue/topic. Without going into detail, suffice it to say that I suspect many people would prefer to see, say, the 10 players above and below them, rather than the top 20, though being able to flip between them would obviously be ideal.
Title: Re: Increase spread of player ratings
Post by: Mic Qsenoch on 08 October 2017, 06:28:11 PM
The leaderboard updates at 0:00 UTC.

Some links you may find useful, if you don't know about them already:
http://dominion.lauxnet.com/leaderboard/ (matches the displayed leaderboard in game, possible to load full leaderboard).
http://dominion.lauxnet.com/live_leaderboard/ (updates your rating immediately after a game is finished).
http://dominion.lauxnet.com/scavenger/?user=Sharur
Title: Re: Increase spread of player ratings
Post by: markus on 08 October 2017, 06:30:49 PM
Mic already posted the nice links.  ::)

As you say, the transformation is completely arbitrary and one could have whatever most people like. An argument for not increasing the spread would be that you can't measure a player's skill accurately. Even now, only when someone is at least 3 levels above somebody else, you can be reasonable sure that they are the better player. (assuming that both players have the same phi - whether the leaderboard should make phi less important is a different story.)