What about ranking / leaderboards ... ?

Previous topic - Next topic

Martin plays Piano

Hi,
I am wondering to find nothing in this forum or within ,,Dominionstrategy" about the future mode of voting, ranking or statistics. There are different ratings to be set when creating new tables, but currently it is not embedded in the beta testing.
However for the testing the ranks might be extraneous, but nevertheless it should be tested as well – and it should be announced how it will work. I assume this has also to start with 1st of January, correct?

- Do we start with zero points and zero games (or is MF data transferred ?)
- What means a rating of +30 or -70 ? (I am not familiar with these levels)
- Is there a new page with leaderboards and statistics ?
- What about daily / weekly statistics with temporary leaderboards ?
- Are all games voted ? – or can you switch off to play non-voted games ?
- The difference in MF between "casual" and "pro" – will this remain ?
- Is there a difference between voting bots games vs. voting games with real players ?
- Is the ranking / level etc. always visible for other players (e.g. when creating new games) ?
- What about quitters – are they punished with minus voting ?
So many questions ...


Stef

Quote from: Martin plays Piano on 29 December 2016, 06:26:13 PM

So many questions ...

We have not fully figured out what we want to do with ranking / leaderboard yet.
Some points in sort of random order...

* We will scale the rating such that everyone is between 0 and 100.
* I don't really want a leaderboard to be important. In my opinion, the only good reason to have a ranking at all is to easily get good matches in the automatch. If you feel a need to find out who is best, play some tournaments.
* We won't rate games against bots or with >2 players.
* You will be able to specify a game is not-rated
* It is possible to "abuse" the familiar cards feature to only get specific cards/kingdoms, so probably games where the familiarity of cards had a large impact on the kingdom have to always be non-rated. This isn't really a problem, because the intended users of this feature aren't all that interested in rated games anyway.
* We will start out by importing the ratings from the "isotropish" leaderboard and use that for the first two weeks of 2017.

E.Honda

Quote from: Stef on 29 December 2016, 07:13:54 PM
* I don't really want a leaderboard to be important. In my opinion, the only good reason to have a ranking at all is to easily get good matches in the automatch. If you feel a need to find out who is best, play some tournaments.

I agree that a leaderboard isn't necesarily reflecting who are the best players, but i think it would be nice to have one nonetheless. Of course if you want to play competitively you can just play all those well stacked tournaments out there, but i think there are also a lot of players who enjoy playing competitively but aren't in the community long enough to know about all those tournaments and might enjoy having a simple leaderboard to climb. I mean, the MF leaderboard didnt really have a good rating system and you could get to the top by just playing like 10 games against the same Account, but to me it was always fun trying to climb a bit while playing the normal matchmaking games.
So I think it could be good to keep a lot of players motivated for the matchmaking queue.

AdamH

Quote from: Stef on 29 December 2016, 07:13:54 PM
* We won't rate games ... with >2 players.

I really don't think this is a good idea. Even if it's a separate leaderboard, there needs to be some acknowledgement that 2p dominion is not the only competitive format for the game.

Emeric

Quote from: AdamH on 30 December 2016, 04:47:19 PM
Quote from: Stef on 29 December 2016, 07:13:54 PM
* We won't rate games ... with >2 players.

I really don't think this is a good idea. Even if it's a separate leaderboard, there needs to be some acknowledgement that 2p dominion is not the only competitive format for the game.
You can play dominion more than 2 players ????  ;)

Infthitbox

Quote from: AdamH on 30 December 2016, 04:47:19 PM
Quote from: Stef on 29 December 2016, 07:13:54 PM
* We won't rate games ... with >2 players.

I really don't think this is a good idea. Even if it's a separate leaderboard, there needs to be some acknowledgement that 2p dominion is not the only competitive format for the game.

It doesn't seem like that much more work to have separate rankings for 2p, 3p, 4p..., especially if the rankings continue to be based on TrueSkill. Of course, in a 2p-only environment, I'd prefer TrueSkill not to be used, as it doesn't have any additional utility over say, Glicko2 in that environment.

Also, people crave leaderboards. It drives people to play, questing up the leaderboard and all that jazz. I think from an incentive to play standpoint a leaderboard is a must. It makes it possible for people to set discrete goals for themselves (top 100 being one I had just achieved, albeit not on a consistent basis before I stopped playing for months) and motivate themselves to play. Sure, tournaments are great, but the likelihood that you win one is pretty low (unless you're a top player) and people still like to feel like they've achieved something.


I've just realized that setting a rating cap means you're either not going to use an Elo-based system or you are going to be doing some funky scaling magic. Can't say that I'm too familiar with a system that has a cap on it. 

Bryan

Quote* I don't really want a leaderboard to be important. In my opinion, the only good reason to have a ranking at all is to easily get good matches in the automatch. If you feel a need to find out who is best, play some tournaments.
I disagree strongly. People understand that ladder leaderboards are not who is the best player, but who is the best at laddering.

QuoteAlso, people crave leaderboards. It drives people to play, questing up the leaderboard and all that jazz. I think from an incentive to play standpoint a leaderboard is a must.
I think ladders are very important for online games.

Additionally, I've spent a lot of time thinking about rating systems and the assumptions made by elo, trueskill, glicko and how they make assumptions that aren't entirely reasonable for online games. Given that I've created my own rating system derived from Elo but changing core assumptions.

You can see it working on a real population of duelyst players for ladder and tournament on my website at http://mmr.host/. I obviously think my own solution is great so I am plugging it. As far as I can tell, since I made this website for the community, players at the top end of the ladder have enjoyed competition with one another much more.

Summary:
A. Ladder competition is hugely important to the success of an online game.
B. In my opinion, traditional rating system solutions aren't optimal for these use cases(and that's maybe how anti-ladder opinions were formed?).

Stef

Just to be clear: "I don't want the leaderboard to be important" doesn't imply we won't have one. We most certainly will. I just won't put it in people faces, like displaying it after every game.

amysue102

For me the value of ranking is about finding compatible players... if I'm always playing people a lot better than I am (or vice versa) it's not that fun.

Felendis

#9
Quote from: Stef on 29 December 2016, 07:13:54 PM
* I don't really want a leaderboard to be important. In my opinion, the only good reason to have a ranking at all is to easily get good matches in the automatch. If you feel a need to find out who is best, play some tournaments.
I read this a little while ago, and it has been sticking with me. This gives me a really bad feeling. At best, its poor wording and poor community relations. At worst, this is not going to be the game that the consumer wants to see, its going to be the game that Stef wants to make.

Either way, this makes me very leery of even investing my time here, let alone my money. Lets set aside the fact that leaderboards are very important to me for now, all of this comes off very unpolished and unprofessional, like a hobbyist with no idea how to run a business wanting to make a game.

/sigh

limetime

I think this is more an issue of priorities than of him not caring. He is really really busy right now...



Stef: I think that releasing logs so that others can make leader boards will relieve pressure and open up opertunities for people to make tools to analyze data.

Infthitbox

Quote from: limetime on 01 January 2017, 07:26:48 AM
Stef: I think that releasing logs so that others can make leader boards will relieve pressure and open up opertunities for people to make tools to analyze data.

If that's the case, I'll tentatively volunteer to work on that. I had been doing a fair bit of the old community gokosalvager programming for a while (up to say, 5 months ago when I just quit Dominion). I doubt releasing logs is high up the priority list though, as mentioned.

Seprix

Quote from: Felendis on 01 January 2017, 07:10:44 AM
Quote from: Stef on 29 December 2016, 07:13:54 PM
* I don't really want a leaderboard to be important. In my opinion, the only good reason to have a ranking at all is to easily get good matches in the automatch. If you feel a need to find out who is best, play some tournaments.
I read this a little while ago, and it has been sticking with me. This gives me a really bad feeling. At best, its poor wording and poor community relations. At worst, this is not going to be the game that the consumer wants to see, its going to be the game that Stef wants to make.

Either way, this makes me very leery of even investing my time here, let alone my money. Lets set aside the fact that leaderboards are very important to me for now, all of this comes off very unpolished and unprofessional, like a hobbyist with no idea how to run a business wanting to make a game.

/sigh

1. There will be leaderboards!
2. The gloss is not as important as a functional game. Without a functional game, all the pretty sparkles and graphics in the world aren't going to make the game look good. A 100% working game is the top priority, but it will look better soon.

I can tell you right now, this is already a better launch than MF and especially Goko. The interface is already much much better than MF (even if it doesn't look as polished), and the polish will come.

derg17

Thanks again for your work! Great to hear there will be leaderboards!

I'm not sure what it will feel like to have a leaderboard as a number from 1-100... But I would say that I think it is critical for someone to be able to see clear feedback on how their playing is effecting their rank. (Maybe it doesn't pop up in their face after every game. But after a game I think they should be able to check and see where they now stand in the rankings if they want.) And this should be true for someone who is ranked low, middle and high. With lack of a clear way to see where they stand and an ability to check their progress, I worry people will lack motivation. Even my friends who are lower on the board report back to me when they move up a bit.

It's just a reality that people are highly motivated by rankings. It seems that this motivation should be leveraged in the most clear way possible. Maybe that will work great in what you have in mind! I was worried what you have described would obscure rather than illuminate someone's ranking/standing.

toadhill

I only ever play 2 person games against bots. Always watching my "number" to determine if I am getting better or falling behind. Not really the "leaderboard" which didn't interest me. But without some kind of running score, it becomes pretty pointless for me to keep playing. Really miss the MF version. A LOT!