Obvious Blacklist Abuse

Previous topic - Next topic

Sharajat

So an obvious abuse of every blacklist system is to just blacklist top players you lose to to avoid them.  Should there be a cap on the number of players that can be blacklisted, or will the admins watch for accounts that have blacklisted high level players?

Jacob Marley

This would only be a problem insomuch as it allows people to game the leaderboard, by trying to get easier wins.  Otherwise, it is only limiting the pool of players they can play against.  The problem with gaming the leaderboard, is that it should be hard to climb the leaderboard if all you are playing are low rated players.  I don't know enough about Glicko 2 to tell how much of a problem there is.  At this point I don't think there need to be limits, but if time goes on and it seems that there are a lot of players high in the leaderboard who are only playing low ranked players, then maybe revisit the question.

SkyHard

And here I thought it would be easier to climb the leaderboard playing against strong players. Why? A single win has a great impact whereas a loss not that much.

On the other hand, loosing against a low ranked players will have a big impact (and with some bad luck it is always possible to lose - my opinion as a non pro)

tufftaeh

With "Equal Opponent" you are already guaranteed not to be matched with a player more than 12.5 levels above you. And if you create a table, you can limit the level difference even more (and even allow only much weaker opponents). So there is no need at all to use the blacklist for this. Anyway, as others have already pointed out: You don't get too many ranking points for wins against much weaker players...

JW

If the rankings were badly mis-estimating the win probabilities for players with substantially different rankings, there might be a way to game the rankings system as you describe. But if you never take the risk of playing against anyone better than you, this will stifle your ability to improve. 

gitsticker8

Of course the blacklist can be abused in this way and in others as well.  This is why most popular online multiplayer games don't allow it, but offer instead /report and /ignore features.  I understand people want a blacklist so they can avoid trolls, but the problem isn't the trolls.  The problem is the game design that allows trolls to be so effective with slow-rolling and no option for ignoring their chat. 

This is off topic, but if less time were allowed per turn (5 minutes +10 seconds per action taken for example) slow rolling would be less of an issue and the calls for a blacklist would diminish.

This of course is only relevant if the goal is to actually have a meaningful leader board which I assume is the case.

JW

Anyone who argues the blacklist should be removed because of the potential to abuse it to alter ratings should also provide a description of why the ratings system is doing a poor job of predicting the outcome of certain games. If that were to be fixed, the potential for abuse would go away. I'm much more interested in playing against opponents I enjoy playing with than with the precise accuracy of the leaderboard, and I assume many users are the same way.

gitsticker8

Quote from: JW on 04 May 2017, 08:21:41 AM
why the ratings system is doing a poor job of predicting the outcome of certain games.
Doesn't this have something to do with the game design itself?  It's quite likely that a lower rated player can win just by luck alone (go first, draw 5/2 with witch chapel on the board for example).
Quote from: JW on 04 May 2017, 08:21:41 AMIf that were to be fixed, the potential for abuse would go away.
I don't see how improving the rating system's performance would eliminate blacklist abuse unless I'm missing something.  Could you elaborate?
Quote from: JW on 04 May 2017, 08:21:41 AM
I'm much more interested in playing against opponents I enjoy playing with than with the precise accuracy of the leaderboard, and I assume many users are the same way.
I think you make a very valid point here. What's the point of playing?  To have fun.  But what is the point of a leaderboard?  To provide accurate information that allows players to see how they stack up against others.  If the information isn't accurate (ie open to abuse) then I don't see a point of having a leaderboard.  Inevitably there will be people who rise to the top by abusing the system which discourages people from trying to be competitive in the first place. 

I do think both concerns can be addressed however.  If the ability to troll people were mitigated to a greater extent, we wouldn't need the blacklist. 

JW

Quote from: gitsticker8 on 04 May 2017, 05:14:00 PM
Quote from: JW on 04 May 2017, 08:21:41 AM
why the ratings system is doing a poor job of predicting the outcome of certain games.
Doesn't this have something to do with the game design itself?  It's quite likely that a lower rated player can win just by luck alone (go first, draw 5/2 with witch chapel on the board for example).

The rating system takes into account that the lower rated player is expected to win a certain percentage of games.

Quote
Quote from: JW on 04 May 2017, 08:21:41 AMIf that were to be fixed, the potential for abuse would go away.
I don't see how improving the rating system's performance would eliminate blacklist abuse unless I'm missing something.  Could you elaborate?

Whether you rise or fall in the ratings depends on how you do compared to the rating system's expectations.

Quote from: Philip on 28 April 2017, 12:21:52 AM
Note that you don't rise in level merely for winning, you have to perform above the rating's system's expectation.

Using purely hypothetical examples, if the rating system thinks that a player 15 levels higher than another one wins, say, 95% of the time, when the higher rated player actually wins 85% of the time, then the higher rated player would have reasons to avoid that matchup. If the rating system thinks that a player 15 levels higher than another one wins 85% of the time, when the higher rated player actually wins 95% of the time, then the lower rated player would have reasons to avoid that matchup. If the system is basically accurate, then no one has reason to avoid the matchup.

gitsticker8

Thanks for that explanation, well said.  You're arguing that a perfect matching system with rewards perfectly proportional to the risk of losing is impervious to abuse.  And I agree completely based on your explanation.  I would also argue that the more control you give the player as to who they play against, the less meaningful the leaderboard becomes in general.  Blacklisting effectively limits the player pool.  If my player pool has 10 players and yours has 10 completely different players, should we both be rated on the same leaderboard?  I think that would be a good case for 2 separate leaderboards.  Granted this will likely never happen, but the more blacklisting that goes on, the closer we approach that scenario.

What about the case of a person who blacklists everyone who takes longer than 1 minute for their first turn?  What if it becomes common practice to share blacklists thus alienating players and reducing their opponent pool unbeknownst to them?

One more point about the rematch option since I think it is relevant to this conversation.  I believe rematch games should NOT be rated.  It enables win-trading (intentionally losing to your friend over and over to boost their rating) which is just another way to game the leaderboard.  The only games that should be rated are those that the matching system sets up.

JW

Quote from: gitsticker8 on 04 May 2017, 08:17:57 PM
One more point about the rematch option since I think it is relevant to this conversation.  I believe rematch games should NOT be rated.  It enables win-trading (intentionally losing to your friend over and over to boost their rating) which is just another way to game the leaderboard.  The only games that should be rated are those that the matching system sets up.

The logic here implies that games also should not be rated if you set them up directly with friends. I would expect that the percentage of games set up directly with friends or rematch games that are legitimate (versus intentionally thrown to game the rating system) is greater than 99.9%.

ShuffleIT seems to have hit on the correct solution to this potential issue: "While it's possible to beat up dummy accounts in order to game the leaderboard, this is not a good idea if you're interested in the longevity of your account." http://forum.shuffleit.nl/index.php?topic=1679.msg5888#msg5888

QuoteI would also argue that the more control you give the player as to who they play against, the less meaningful the leaderboard becomes in general.  Blacklisting effectively limits the player pool.  If my player pool has 10 players and yours has 10 completely different players, should we both be rated on the same leaderboard?  I think that would be a good case for 2 separate leaderboards.  Granted this will likely never happen, but the more blacklisting that goes on, the closer we approach that scenario.

Players are already highly segregated based on what times they play (mainly due to different time zones). I expect this far outweighs any effects of the blacklist. And I don't think that people playing Dominion all over the world is a concern for the leaderboard, so this doesn't seem like a concern either.

gitsticker8

Quote from: JW on 04 May 2017, 09:20:56 PM
The logic here implies that games also should not be rated if you set them up directly with friends. I would expect that the percentage of games set up directly with friends or rematch games that are legitimate (versus intentionally thrown to game the rating system) is greater than 99.9%.
Yes, I believe custom games should not be rated either.  Only games that are set up by the matching system.  While I agree that .1% is not a big number of people abusing the system, it doesn't take many rotten apples to spoil the barrel because those are the names you will always see at the top of the board.  Again, the more power the players have to define their opponent pool, the less meaningful the boards become.

Quote from: JW on 04 May 2017, 09:20:56 PM
ShuffleIT seems to have hit on the correct solution to this potential issue: "While it's possible to beat up dummy accounts in order to game the leaderboard, this is not a good idea if you're interested in the longevity of your account." http://forum.shuffleit.nl/index.php?topic=1679.msg5888#msg5888
This is indeed the correct response, I agree.  I just wonder if it is all teeth and no bite? There's not even an option to report, how can we expect account bans when there is no /report functionality? Have there been any account bans yet?

Quote from: JW on 04 May 2017, 09:20:56 PM
Players are already highly segregated based on what times they play (mainly due to different time zones). I expect this far outweighs any effects of the blacklist. And I don't think that people playing Dominion all over the world is a concern for the leaderboard, so this doesn't seem like a concern either.
Good point, and something I forgot to bring up.  Separate leader boards for different regions/timezones would be ideal. I'm just not sure if the playerbase is large enough to support it.  So yes, people playing all over the world is a concern (of mine) for the leaderboard.

tufftaeh

Games for the Dominion League (http://forum.dominionstrategy.com/index.php?board=60.0) are very certainly some of the most legitimate around, and those include a lot of rematches. It would be completely stupid to exclude them from the rankings.
Currently, there seems to be no bad apple in the top 20, so your theory of inevitability has not yet manifested at least. Or such accounts have actually been removed...? :)

gitsticker8

On the contrary.  Games in a league or part of a tournament setting should not be rated for the same reasons I already stated.  I disagree that this a completely stupid idea.  Take a look at any other widely popular game with a rating system.  Starcraft for example, has a rating system that matches you with opponents of near equal rating.  Any custom games (including tournaments) are not rated.  I don't think this makes Blizzard completely stupid, quite the opposite.

AdamH

I'm just gonna weigh in here and say that the leaderboard would be far more legitimate if only automatch games counted towards it -- and any kind of blacklist wasn't taken into account. I agree pretty strongly with this statement:

Quote from: gitsticker8 on 04 May 2017, 08:17:57 PM
I would also argue that the more control you give the player as to who they play against, the less meaningful the leaderboard becomes in general.

On the other hand, I don't think that means the best solution is to make the pro leaderboard this way. If there was money on the line for leaderboard position then maybe I'd say it was worth it, but I don't think anyone takes the leaderboard seriously enough that sacrificing this much fun is worth it.

I've never really cared about the leaderboard. I think that implementing all of the cards correctly, without bugs, and displaying all of the relevant information to the user, and all of the other stuff I constantly complain about, would have a more significant impact on how legitimate the leaderboard is than what's being discussed here.

gitsticker8

Quote from: AdamH on 05 May 2017, 01:47:33 AM
I don't think anyone takes the leaderboard seriously enough that sacrificing this much fun is worth it.
I don't see how separating custom tournament games and rematches from the leaderboard rating sacrifices fun.  Could you elaborate?

Quote from: AdamH on 05 May 2017, 01:47:33 AM
I've never really cared about the leaderboard. I think that implementing all of the cards correctly, without bugs, and displaying all of the relevant information to the user, and all of the other stuff I constantly complain about, would have a more significant impact on how legitimate the leaderboard is than what's being discussed here.
Point taken.  I think that people who are in the competitive scene and play regularly with the best of the best are less likely to care much about the leaderboard because they are so often measured that they have a pretty good sense of where they stand on the totem pole.  I'm not making any assumptions about you, but I care quite a bit about the leaderboard because it's the only measure I have for myself as to how good of a player I am.  I'm not part of any leagues or tournaments (yet) and I'm new to the forum.  So from my perspective, a shaky leaderboard disincentivizes me from being competitive.  I do agree with your point that there are other glaring issues that are more pertinent, but the blacklist is imminent, hence the recent discussion.

Geronimoo

Quote from: AdamH on 05 May 2017, 01:47:33 AM
I've never really cared about the leaderboard. I think that implementing all of the cards correctly, without bugs, and displaying all of the relevant information to the user, and all of the other stuff I constantly complain about, would have a more significant impact on how legitimate the leaderboard is than what's being discussed here.
Since all players are affected the same way I don't see how this can have a significant impact on the leaderboard.

AdamH

Quote from: gitsticker8 on 05 May 2017, 03:43:51 AM
Quote from: AdamH on 05 May 2017, 01:47:33 AM
I don't think anyone takes the leaderboard seriously enough that sacrificing this much fun is worth it.
I don't see how separating custom tournament games and rematches from the leaderboard rating sacrifices fun.  Could you elaborate?

It's mostly rematches I'm talking about. Now people who want all of their games to count towards the leaderboard are forced with the choice of playing against their friends or having their games count. They can't have both.


Quote from: Geronimoo on 05 May 2017, 10:17:04 AM
Quote from: AdamH on 05 May 2017, 01:47:33 AM
I've never really cared about the leaderboard. I think that implementing all of the cards correctly, without bugs, and displaying all of the relevant information to the user, and all of the other stuff I constantly complain about, would have a more significant impact on how legitimate the leaderboard is than what's being discussed here.
Since all players are affected the same way I don't see how this can have a significant impact on the leaderboard.

But all players are not affected in the same way. The bugs are not well-documented and there is nothing on the client to let players know which cards don't work properly and how they don't work properly. Players that have a deeper knowledge of the shortcomings of this software have an in-game advantage because they can more effectively play around them.

And this doesn't take into account at all any of the games that just crash and one player is forced to resign a game they might have won due to something other than skill.

Ingix

Quote from: AdamH on 05 May 2017, 01:47:33 AM
I'm just gonna weigh in here and say that the leaderboard would be far more legitimate if only automatch games counted towards it -- and any kind of blacklist wasn't taken into account.

I'm not sure if the game logs/database contains this data. It would be interesting to know how many ranked games are purely based on the server matchmaking and how many due to rematch/explicite matchup

Quote from: AdamH on 05 May 2017, 01:47:33 AM
I've never really cared about the leaderboard. I think that implementing all of the cards correctly, without bugs, and displaying all of the relevant information to the user, and all of the other stuff I constantly complain about, would have a more significant impact on how legitimate the leaderboard is than what's being discussed here.

While I agree that there are many rules bugs that still exist, I don't think it has any significant impact on games, because they are on the fringes of what usually happens during play. I'd say that for the *average* player the games played here are nearer to the theoretical "absolute conformation to the rules" than what happens in RL play, because your Actions, Buys and Coins are counted for you, Landmarks like Tomb, Aqueduct and others are automatically handled for you and Enchantresses are not forgotten  :)

AdamH

Quote from: Ingix on 05 May 2017, 03:19:28 PM
While I agree that there are many rules bugs that still exist, I don't think it has any significant impact on games, because they are on the fringes of what usually happens during play.

About a week ago I decided that I was going to remove all of the cards with visual or gameplay-related bugs from my familiar cards list so that I wouldn't have to play with them until the bugs were fixed. I was just so tired of drawing a Curse, not understanding why I had a Curse in my deck, then going back in the game log to 3 turns ago when my opponent had a Swamp Hag out and I was given no indication of it at all.

I realized two problems with this while attempting to do it:

1. The only lists of bugs that exist for the public to view (that I am aware of) are the pinned threads at the top of the Bug Reports boards on these forums. There is no indication of how up-to-date they are (these forums don't show if/when posts have been edited for reasons I don't understand) and I know that they haven't been updated in quite some time.

2. Just by the bugs that are currently posted in those threads, I realized that I'd be taking away such a large chunk of the kingdom cards (it was over 55) that it made me feel *really* bad. I'm not going to give a precise number here because who actually knows what the current outstanding bugs are and how much the affect play, but suffice it to say that if this number was calculated for-realsies, I would guess that at least 55% of games are affected by this in some way.

So saying that it's on the fringes of what happens during play? Man I just don't buy that. If it comes up in 1% of games then it's not good enough, and if one kingdom card has a problem, it shows up in 2.5% of all games. Sure, maybe the bug won't enter play in every game but, there are three cards (the duration attacks) that will ruin every game they are in IMHO (more on that below) and over 9% of all games will have at least one of them in it. So yeah I'm just not convinced that card bugs and visual bugs don't have an effect on the leaderboard.

The big takeaway from this, which I'll kind of gloss over because I don't want to go too far off topic, is:

There is no indication anymore that the devs are even keeping a backlog of the issues remaining with their software and/or that it's up-to-date. This would go a long way towards making their community feel like they're being heard at the very least, but also towards letting people know which cards they can feel confident playing with and not running into bugs. In fact, due to the lack of any kind of formal tutorial being present in the software, the Dominion rulebooks are the closest thing people have to a guide on how to use the software, so a readily-available list of how the online version deviates from the rulebooks should definitely be a thing. This includes stuff like how the Black Market deck is constructed; wording changes for reprintings that haven't been reprinted, but are present on the online client, which affect gameplay (I'm thinking of Scheme but I know there are more); and obviously known bugs.


Quote from: Ingix on 05 May 2017, 03:19:28 PM
and Enchantresses are not forgotten  :)

I think you're kidding here, right? I mean, there is no visual indicator to show that your opponent has an Enchantress out other than hovering on their name. I have never once forgotten about an Enchantress IRL and I'm struggling to remember a single time online when I played an Enchanted card knowing it was going to be Enchanted before I saw that it didn't do what I expected.

Yes, what happened here is correct by the game rules, but the game rules also state that when playing an Action card, you're supposed to give some indication to your opponent of what you're doing, which is not happening in a good enough way on the online version. Visual bugs like this are more than enough to take away from the legitimacy of what happens in a game of Dominion, which contributes to making the leaderboard less legitimate.

The three big offenders here (the duration attacks: Haunted Woods, Swampy, and Enchantress) together are represented in over 9% of all games of Dominion -- this is what I was talking about before.

------------------

In any case, all that was just to support the argument that card bugs and visual bugs do have an effect on the legitimacy of the leaderboard. In spite of that, I don't think that ultimate integrity of the leaderboard is worth what would have to be sacrificed in order to achieve it, mainly because there's little upside to having a leaderboard with that kind of integrity.

Ingix

Let's be clear: The duration attacks not being shown when they are out is a mayor PITA and a source of confusion (that's why the Enchantress had a smiley in my posting: It is taken care of by the game, but not being shown has caused lot's of confusion).

But if in a ranked game you don't realize that a curse giver is in the supply, you didn't realize that your opponent bought one, didn't realize that he played one, then, yes, again, it would have been nice if we had some indication that it is currently affecting game play, but *please* pay some attention to what happens when it is your opponent's turn. The advantage of the online game, not needing to physically shuffle cards etc. has the disadvantage that some things, like curses being handed out, happen without your 'help'.

Quote
So saying that it's on the fringes of what happens during play? Man I just don't buy that. If it comes up in 1% of games then it's not good enough, and if one kingdom card has a problem, it shows up in 2.5% of all games. Sure, maybe the bug won't enter play in every game but,

First, I was talking (maybe not clearly enough) about real rules related problems, not visual things, like the duration attacks. Things like Scheme+Herbalist (http://forum.shuffleit.nl/index.php?topic=1594) or Kings Court +Scheme (http://forum.shuffleit.nl/index.php?topic=1480).

Second, even if Scheme is in the Kingdom, the problems don't usually come up, because they require some enabler card  (like Herbalist or Kings Court). That's what I meant with 'on the fringes'.

So in my opinion, just having a problem card in the Kingdom is absolutely not the same as being any rules problem to be expected.

AdamH

Quote from: Ingix on 05 May 2017, 08:42:53 PM
Let's be clear: The duration attacks not being shown when they are out is a mayor PITA and a source of confusion (that's why the Enchantress had a smiley in my posting: It is taken care of by the game, but not being shown has caused lot's of confusion).

But if in a ranked game you don't realize that a curse giver is in the supply, you didn't realize that your opponent bought one, didn't realize that he played one, then, yes, again, it would have been nice if we had some indication that it is currently affecting game play, but *please* pay some attention to what happens when it is your opponent's turn. The advantage of the online game, not needing to physically shuffle cards etc. has the disadvantage that some things, like curses being handed out, happen without your 'help'.

This seems to be the main place where we disagree. I can go on and on about why I feel the way I do but what you've said doesn't change my mind. I already pay attention to what's happening on my opponent's turn.

Unless I'm mistaken, hovering over a player's name is not documented or shown to the user in any way (the only people who can be expected to know about it are people who come to these forums and find places that tell you to hover over the names for this info -- is that a skill you want tested on the leaderboard?), so really the only reliable indication I have that a duration attack is by reading the game log.

That being the expectation just doesn't fly for me. The UI needs to be better (and it will be better, I believe the devs have said they're going to change this), and until it is, I'm not convinced. I mean, I used duration attacks as examples because to me, they are the cards that are clearly the most broken and unplayable by far.

As for actual mechanical/gameplay bugs, I don't really see the point in discussing the impact on the total games played without actually having a list of these bugs in the current release and crunching the numbers on it. Without that list, we can guess at the impact, and obviously my guess is higher than yours, but it's all just a guess and I won't be convinced of anything by anything other than numbers at this point.

gitsticker8

Can we keep the thread on the topic of blacklist abuse?  Or maybe a moderator can split this into 2 separate threads?

tufftaeh

Quote from: gitsticker8 on 05 May 2017, 01:25:48 AM
On the contrary.  Games in a league or part of a tournament setting should not be rated for the same reasons I already stated.  I disagree that this a completely stupid idea.  Take a look at any other widely popular game with a rating system.  Starcraft for example, has a rating system that matches you with opponents of near equal rating.  Any custom games (including tournaments) are not rated.  I don't think this makes Blizzard completely stupid, quite the opposite.

I have no idea about Starcraft. It's quite possible that there are enough idiots playing Starcraft who would "game the leaderboard" if it was possible so it is actually necessary to limit the Starcraft ranking to random opponents. I was talking about Dominion. And I stand by my opionion that it would be completely stupid to remove the games between the absolute best players from the ratings just because they played multiple subsequent games in one league match. I guess we'll have to agree to disagree. :)

gitsticker8

There are always idiots or trolls.  This is just the way of the internet.  "If you build it, they will come."  so to speak.  It's just a matter of how you manage them.  If you give them power, they will exploit it and ruin it for everyone to the extent that it is possible.  Dominion is not in any way immune to this.