Leaderboard

Previous topic - Next topic

serakfalcon

Hi, I noticed the announcement that there is ratings and a leaderboard (yay). The leaderboard only shows the top 20, is there a link somewhere to show the whole list?

tufftaeh

The leaderboard displays you (after you have played one rated game and a subsequent update at 0:00 UTC), your friends and the top 20 players. See this post: http://forum.shuffleit.nl/index.php?topic=1679.0

serakfalcon

Quote from: tufftaeh on 01 April 2017, 12:47:24 PM
The leaderboard displays you (after you have played one rated game and a subsequent update at 0:00 UTC), your friends and the top 20 players. See this post: http://forum.shuffleit.nl/index.php?topic=1679.0

So is there going to be a way to see more than the top 20??

Stef

Quote from: serakfalcon on 01 April 2017, 04:27:16 PM
So is there going to be a way to see more than the top 20??

Most likely not in the client. It should show everyone you're interested in. If there is a good demand for a full leaderboard we could put it somewhere on the web though, or I could generate an export of the ratings history database at some point if someone is interested in creating statistics on that.

kieranmillar

If you don't want the complete leaderboard in the client, what about showing those in the positions next to you? Like the 5-10 above and 5-10 below, or whatever works best.

serakfalcon

I think a few extra pieces of information would be useful (who is close to you, how many people have your ranking, how many people there are below you), it would be nice to have a full list but most people using it would probably looking for that info anyway.

The top 20 is only 'useful' information for people who are either in the top 20 or expect to be, however I'm sure anyone playing ranked games would like to know whether or not they are improving.

werothegreat

So I won't be able to see myself on the leaderboard until after a midnight update?

limetime

Is there a reason it only updates once everyday?

Watno

Quote from: werothegreat on 02 April 2017, 09:03:16 PM
So I won't be able to see myself on the leaderboard until after a midnight update?
yes

Quote from: limetime on 02 April 2017, 10:43:45 PM
Is there a reason it only updates once everyday?
yes, the Glicko-2 rating system is designed to work best when the players play multiple games between updates.

markus

It's true that you want to have the rating period at least one day (in the paper it says best at 10-15 games per period). But it would be possible to show an update during the day, which would be the (exact) prediction of your level at the end of the day, if you stop playing now. For details see my post here.

I kind of like that there's one published leaderboard per day. And my suggestion above would not affect the ratings at all (e.g. whether you first win against someone and then lose or the other way around is irrelevant within the rating period). But I can see that some people want to get some instant feedback.

markus

I'm adding this thought to the "general discussion" of leaderboard: I would allow to have rated games against the bot, if they are drawn randomly (like in "find a game"). The bot would then also get a rating - and could be shown on the leaderboard.

Personally, I only end up in bot games now, if I load a game or want to try something out and typically resign on them. And of course they shouldn't be rated. Also people shouldn't be able to select specific kingdoms to game the bot.

When the bot is not great like now, it will just have a low rating and beating it will not get you to the top of the leaderboard. But apparently there are some people who only/mostly want to play bot games and you could provide them with a rating, which is more informative than level 20.

And you would get feedback on your future bot improvements, when its rating rises.  ;)

Bbl

Quote from: markus on 05 April 2017, 08:58:51 AM
I'm adding this thought to the "general discussion" of leaderboard: I would allow to have rated games against the bot, if they are drawn randomly (like in "find a game"). The bot would then also get a rating - and could be shown on the leaderboard.

Personally, I only end up in bot games now, if I load a game or want to try something out and typically resign on them. And of course they shouldn't be rated. Also people shouldn't be able to select specific kingdoms to game the bot.

When the bot is not great like now, it will just have a low rating and beating it will not get you to the top of the leaderboard. But apparently there are some people who only/mostly want to play bot games and you could provide them with a rating, which is more informative than level 20.

And you would get feedback on your future bot improvements, when its rating rises.  ;)

I love this idea. It would give all these AI only players out there a meaningful way to measure their skill, while providing useful data if they opt to play against human players.

The way I understand the rating system, this should really be not gameable, as the Bot is quite likely to have a very low level score. It also reflects the actual Dominion skill better: Someone who manages an 80% win ratio against a bot in full-random is probably a better player than most newcomers.

The effort to implement this feature should also be rather low compared to its value, so I would be really happy if this suggestion finds its way into the game.

Stef

Quote from: markus on 05 April 2017, 08:58:51 AM
I'm adding this thought to the "general discussion" of leaderboard: I would allow to have rated games against the bot, if they are drawn randomly (like in "find a game"). The bot would then also get a rating - and could be shown on the leaderboard.

I don't think I like this idea. At least on the old site I think the isotropish leaderboard was significantly worse when it still included bot games, and the switch to not including them was made consciously.

The problem with the bots rating is that it would still be inflated, because of all the new players playing against it, and also because of people not caring enough to finish their game.

Also... the bot just isn't very flexible. If you play a lot against bots, you get much better at beating bots then you are at actually playing dominion. While we could measure this skill, I don't think we should mix it up with the regular leaderboard.

Quote from: markus on 05 April 2017, 08:58:51 AM
And you would get feedback on your future bot improvements, when its rating rises.  ;)
That would certainly be an upside of this idea.


If there is a real demand for it we could make a separate leaderboard just for bot games.

markus

I think that you get rid of some of the potential problems mentioned by only using games against bots, where I explicitly say that I want a rated game and I don't have any influence on the kingdom other than through  my subscription.

Then, people might not care less about finishing that game properly than against a human opponent. And there are actually uspides (e.g. if you have to stop playing immediately, you can finish later, whereas it would be a loss against a human opponent; and the bot never has connection troubles / slow plays you).

If you want to be conservative, you can reduce the weight of bot games, e.g. 10 wins/losses against the bot count the same as 1 win/loss against a similar ranked human.

Rating inflation due to newcomers is a valid concern, although it has some upper bound as well. (With the current settings you can't get above mu=3 even with a 100% win rate against a lot of mu=0,phi=2 players. And that is an extreme case, because the bot is assumed not to lose (to better players). I expect the top 20 to move towards mu=6 instead.)

I think the most valid concern is the getting to know and exploiting the bots general weaknesses. At the moment this wouldn't matter at all for the top of the leaderboard. But if the bots get better and rise in their rating this skill could be valuable but shouldn't improve your position on the leaderboard (e.g. if I knew that they don't go for (more or less complicated) 3-piles, I could adjust my play and win more often than against some random opponent)

Ingix

I would definitely support ranked play with Bots, at the moment and in the foreseeable future with a separate leaderboard.

As mentioned, this has advantages for new and less-skilled players, who can get measurable results that (hopefully) get better over time. Even more important would be to actually see what effect changes in AI (big or small) have. Since the bot (or in the future, different bots) would probably play lots of ranked games, the ranking would be pretty accurate relatively fast. I think this would create very valuable data for testing/evaluating AI changes.