3P ratings when one player resigns?

Previous topic - Next topic

AdamH

Quote from: Accatitippi on 01 June 2017, 12:33:57 PM
Adam's solution has merit because it's simple, but I personally wouldn't enjoy it much. I play to win, but I mostly play to play, and being forced to end an open game with no clear winner would annoy me almost as much as ending it with a arbitrary one (as it is now).

I agree with you, the option to continue the game with a bot (or as a 2P game) should be offered to the remaining players in the interest of fun, but whatever happens there should have no effect on the leaderboard result for that game.

Accatitippi

I don't know. I mean, by entering a 3p game you accept that there may be some "kingmaking/helping" interference in your game due to the inherent three-playerness. This even considering that Dominion has rather little "politics".
The fact that this interference is controlled by a bot rather than by some other player would not be relevant to me.


I would not complain about kingmaking, as long as the bot is written fairly. What the bot does to hinder me could as easily be done by a player, and at least the bot is more transparent.

And again, I'm not the target audience, so maybe I should just shut up.  ;D

Ingix

Only Stef would know, but is it even possible to convert a 3P game to 2P for continuation?

If it is possible, what are the disadvantages? I know that the number of cards in the victory piles are now incorrect and certain cards shift in power, but is this considered a big problem? Or is it simply the fact that a 3P game is not zero-sum and that this is why people prefer it over 2P, and continuation as a zero-sum 2P is what they don't like about it.

AdamH

Quote from: Accatitippi on 01 June 2017, 03:04:09 PM
as long as the bot is written fairly. What the bot does to hinder me could as easily be done by a player, and at least the bot is more transparent.

"as long as the bot is written fairly" is something that is almost impossible to define, and even less possible to enact or enforce. It's so easy to see a leaderboard result you don't like and feel really bad because the bot decided to pivot into a thing that totally screws you over -- and be totally justified in being upset about that. The result you get from the continuation of the game can in no way reflect the skill required to do well in 3P games of Dominion (it requires you to be familiar with the AI of the bot so that you can hedge against certain people resigning, etc. Yes, it's transparent but it has nothing to do with Dominion skill which is all that should be represented on the leaderboard).

Example: bots end games on a loss all of the time. You and I are playing a game and Joebob resigns, so now we're playing against a bot. Joebob was playing Big Money, but the bot suddenly starts buying the low pile cards until the game is over and the bot loses. Obviously this is an extreme example, and yes the bot was making legal plays that anyone could make, but I feel like assigning leaderboard results to this kind of thing requires that you have a bot that isn't going to do stuff like this, ever. "Stuff like this" is a huge slippery slope, and "ever" is basically impossible to accomplish. It's just a different thing in my mind to have a human player do this and to have a bot do this -- you can blacklist that human, or you can expect to climb high enough above them on the leaderboard to not have to play them anymore, but the bot will always be there.


Quote from: Ingix on 01 June 2017, 03:31:00 PM
Only Stef would know, but is it even possible to convert a 3P game to 2P for continuation?

If it is possible, what are the disadvantages? I know that the number of cards in the victory piles are now incorrect and certain cards shift in power, but is this considered a big problem? Or is it simply the fact that a 3P game is not zero-sum and that this is why people prefer it over 2P, and continuation as a zero-sum 2P is what they don't like about it.

In terms of what to do for-funsies after the game, there is no perfect way to continue the game as a 2P game when it started as a 3P game. Which option is less bad depends a lot on the kingdom and the state of the game.

The best general option, meaning the one that I would argue should be the default, is to have the resigned player skip all of their turns for the rest of the game. They still gain Purples and other junks, but they don't play any cards or buy/gain anything. You have to worry about corner cases like skipping them for naming cards for Envoy/Contraband/etc. but this is certainly doable from a software perspective. Why do I think this is best? Because it does resemble a component of skill in 3P games -- in a few IRL tournaments I've played in, I'm in 3P games where one person is obviously very new to the game and it's clear early on that they're not going to accomplish much; I have to adjust my strategy (build more) to account for the fact that I can't expect them to get very many Provinces, and I have to be a lot more careful about piles since they can just randomly decide to start buying a card because they saw other people do it and poof, the game is over.

But maybe it's turn 1 of the game and that player disconnected, forcing them to be kicked -- in that case it's probably better to just adjust the number of cards in each victory-card-pile, VP tokens on Landmarks, etc. and just continue the game from there. Maybe it's worth it to offer this as an option to the remaining players if it's still T1 or T2, and have this happen only if all players agree. I guess this is nearly the same as restarting the game with a selected kingdom -- so I guess that shouldn't show up on the leaderboard if that's the way it ends up being implemented under the hood.

My opinion? I think it's a long time before there aren't more important things to do. If the players really want to play that kingdom as a 2P game, there's a chat box and people can just start a new table.

SirDagen

I am voting for bot replacement and I am looking forward to it for a long time.

When I want to resign it's because I lost interest in the game, but often the other players want to play out their win and get angry if I resign, so I am in a uncomfortable bind. Bot replacement is something I am looking forward to happening for a long time.
Greetings,
SirDagen

twasa

I played a handful of rated 3p games this week and think of those I won, most were from resignations and me being in a lead with vp.

AdamH's suggestion sounds practical, with the longer term solution being all players agree on the order of players. That may still be difficult, with some of the players who complain about how unlucky they have been, as opposed to the opponents' luck. Still, probably the best option.

One other option to discourage resigning, may be to let the player get a temporary "resignation demerit". The matching options could then include the option to exclude such players. Maybe even by default. One can then choose whether to risk being matched with someone who had recently resigned. If the same player resigns again during the period, extend it for longer.

Although some support continuing the game with something like a bot replacement, I would hope that would be voluntary, with no penalty for declining to continue.

Accatitippi



"Is built fairly" is easy to define, and means: doesn't collude with other players. I mean, that it must not say: "if the higher-ranked player (or the most handsome, or the most Dutch, whatever) is in the lead, buy Province". It's obvious but worth mentioning.
As long as it's as likely to screw you as it is to screw me, then I'm fine with it.
If I'm playing three player, I must be ready to get screwed - by bot or by human.

Of course you'll want a better bot than our Rattington in that position. But you'll want a better bot than Rattington in any position.  ::) :P

QuoteI feel like assigning leaderboard results to this kind of thing requires that you have a bot that isn't going to do stuff like this, ever.
But humans do stuff like this all the time. I play a good deal of 3-4p IRL, and I will take the last Province to ensure a second position, or push the piles to dangerous levels to get something I really need to make an unlikely comeback. In short, I will interfere even if I'm not in close contention.
So why set an extra-high bar for bots?

I mean, I can see your argument working against 3p dominion in general, but I don't see how it works against bots in particular.

As a rescue mechanism, I believe that replacement by bot wouldn't be problematic in the vast majority of games - even using Rattington would maybe be ok in the meantime, but I'm less sure - I haven't played Big R in a long time.

QuoteIt's just a different thing in my mind to have a human player do this and to have a bot do this -- you can blacklist that human, or you can expect to climb high enough above them on the leaderboard to not have to play them anymore, but the bot will always be there.

Well, the bot can always get better. We can surely agree that your example is very extreme. An OK bot wouldn't do that, and an OK human wouldn't do much better than the OK bot in terms of non-interference.
It may be a slippery slope, but then you're already sliding when you start a 3p game.  :)

AdamH

"tl;dr": We don't include bot games on the 2P leaderboard -- what are the reasons for that, and why should they be different than 3P games?

I think what you want is to have it be so that the strategy for winning 3P games will not be affected by the possibility of someone resigning and a bot replacing them, and you're saying that with a sufficiently good bot, this is theoretically possible.

My stance is that I don't think it's even theoretically possible, but I also think it's impractical. I also believe that even if you had a bot that you claim would do this, the burden of proof is on the person arguing for the bot. Dominion is a game meant to be played with people, the leaderboard is meant to reflect the skill of a player against other players, not against bots. The only real difference between your stance and mine is that you want to try and extract this extra result between the two players who didn't resign based on a 3P game, partially against a bot, and what I'm saying is that that result can't be legitimate unless you show me that it is.

Bots can do any legal play, people can do any legal play; but the bot's behavior is deterministic while the person's play is not (otherwise I don't think you can possibly make an argument that the bot will preserve the integrity of the game). These are nowhere near the same thing, no matter how sophisticated you make that bot -- in order to convince me that a bot will serve this purpose, I'd have to see that the best strategy for winning on every single possible kingdom is unchanged by putting in this bot instead of a human player (by accounting for the possibility that either opponent could resign and be replaced with this deterministic bot). This is impossible to do in a practical sense, and if it were possible, I would probably stop playing Dominion.

What if the bot wins or gets second? What if the player who resigned wasn't far behind? Should only the relative result between the two human players be considered? Playing for first place and playing for second place are definitely not the same thing in all kingdoms, I don't think you can call that result legitimate in terms of the leaderboard no matter what the bot does, but any bot that continues play for a player not mathematically eliminated from coming in anything but last should have the possibility of this.

markus

In the short run, I think you want to have all 3:

1) Notify players that they should complete multiplayer games, because resigning ends the game for everyone.

2) Punish the rating after resigning, in case people care about that.

3) Remove them from the standard settings automatch after the third resignation. Delete one such strike after say 20 finished games or 2 weeks.

AdamH

Quote from: markus on 02 June 2017, 12:48:54 AM
3) Remove them from the standard settings automatch after the third resignation. Delete one such strike after say 20 finished games or 2 weeks.

I think this is entering into dangerous territory, especially when so many people are having trouble with the client and being forced to resign games. This kind of thing should really be evaluated on a case-by-case basis IMO.

Accatitippi

markus, your third bullet is probably way too harsh. :) Maybe make that if you get 3 resignations in a 48-hour period, you may not play 3p in the rest of those 48 hours? It may still be a bad idea altogether. Incidentally, how often are people forced to resign right now? I have all sorts of connection issues but I always manage to reconnect without problems.

Adam, we don't include bots in the leaderboard because there is a big slice of the population playing only bots, and that might skew the ratings. But in principle I wouldn't be against that, but there is the problem of the bot being really bad as it is now.

Then I don't understand what determinism has to do with the issue at hand. Is the problem that the bot becomes predictable? A lot of players are quite predictable as well, at least for better players that know them well. And anyway, a bot need not be necessarily predictable. Most good chess bots are non-deterministic, for example.

Regarding the relative placement of the players and bot, I don't really know how the algorithm works.
But maybe you could secretly rank the bot, and if and only if the bot beats you, count that as a loss relative to the bot. Anyway, when was the last time somebody in contention resigned on you? It won't be very common.

But let me change my stance and argumentate a bit more aggressively instead of defensively. Your proposal of discarding the result of the two players who did not resign will discard a lot of valid results, if resigning stays as common as it is now - and we are not sure we'll be able to do much to reduce it. In a lot of games with two similarily-skilled players and one weaker player, most games won't be able to help ranking the two good players relative to each other.
And for the player in second place the games might even evolve in a game of "stay in contention, screw the third", hoping for him to resign. The player in the losing position always has an amount of kingmaking (or kingvoiding?) power that is bigger than what can normally happen in any game of 3p Dominion.

In my view this leads to problematic situations being more common than if we allow bot replacement, given that players tend to resign when they are hopelessly losing.

Bot is problematic when it has a good shot at first or second place.
Voiding the result is problematic when the resigning player is hopelessly losing, and has the possibility of voiding half the win for the player who is hopelessly winning.
Since resigning mostly happens when a player is hopelessly losing, bot is less commonly problematic than the alternative.

The voiding system is quick and more fair than what is in place now, so I'm fine with that in the meantime. But I believe it's inferior to having the possibility of playing it out with a bot.

I think the ideal solution would be:
Player A resigns. B and C get a window with three choices:
Resign as second place
Void the remaining game
Play it out with a bot.

If either one resigns, we have a clean result that everybody agrees upon. If both void, we have a clean result that everybody agrees upon. If either player wants to play it out, then they should do so, and that game should be ranked, in the interest of the leading player.
If both resign, uh, they all unlock Saboteur as an easter egg, and you count that as a void.

This way everybody is happy: the leading player is never forced to forfeit by an stubborn opponent, the second place guy can get it over with quickly if they want to, an early resign can safely end in a void if everybody is fine with that, and the bot gets to play a few games and is happy.

markus

Quote from: Accatitippi on 02 June 2017, 09:37:19 AM
markus, your third bullet is probably way too harsh. :) Maybe make that if you get 3 resignations in a 48-hour period, you may not play 3p in the rest of those 48 hours? It may still be a bad idea altogether. Incidentally, how often are people forced to resign right now? I have all sorts of connection issues but I always manage to reconnect without problems.

Numbers were suggestive.  ;) I'm not playing 3P games, but I can see that it is really annoying if the game just ends half way. If you start a 3P game with strangers, you should commit to almost certainly being able to finish it / be willing to play it out even if you have lost.
I'm not sure which troubles with the client Adam has in mind, but it has never happened to me that I couldn't reconnect to a game that was still running. (I do sometimes have to reconnect.) But even if it was the case that for someone say 1/3 of games can't be finished due to the client's problem. Wouldn't it be better to sort those people out and not make additional players miserable? (in addition to fixing the cause of the problem, but I don't think these problems even exist)

Quote from: Accatitippi on 02 June 2017, 09:37:19 AM
Regarding the relative placement of the players and bot, I don't really know how the algorithm works.
But maybe you could secretly rank the bot, and if and only if the bot beats you, count that as a loss relative to the bot. Anyway, when was the last time somebody in contention resigned on you? It won't be very common.
A bot that replaces a resigned player should always come last in the end, i.e. you win against the resigned player and the relative position with the 3rd player matters, but not where the bot ends up.

That's a bit off topic, but I've suggested in the past that there should be rated bot games. The main leaderboard could still exclude those games. Even if the bot is bad, it's useful to know that the win rate has improved from 70% to 80%. That kind of feedback is missing for people that don't play rated games. Rating the bot shouldn't be difficult, such that once you have different bots a comparable bot can be chosen.

Quote from: Accatitippi on 02 June 2017, 09:37:19 AM
I think the ideal solution would be:
Player A resigns. B and C get a window with three choices:
Resign as second place
Void the remaining game
Play it out with a bot.

If either one resigns, we have a clean result that everybody agrees upon. If both void, we have a clean result that everybody agrees upon. If either player wants to play it out, then they should do so, and that game should be ranked, in the interest of the leading player.
If both resign, uh, they all unlock Saboteur as an easter egg, and you count that as a void.

I agree with that. (I would still discourage resigning, unless there's unanimous agreement upon ranks.)

Ingix

#27

Sorry, the different quotes previously here ended up totally incorrect representing who said what (because I incorrectly moved quote tags around). I just leave my new contribution, it is on response to AdamH suggesting a "do-nothing" bot taking the place of the resigning player.


The idea to have a "totally dumb bot" that just does nothing occurred to me as well, but as usual the number of corner cases is probably so big that it becomes a mayor undertaking to get them all right. Besides the "name a card" things, what do you do about Masquerade or Possession (Player A can posses player B, but player B can only possess the dumb bot)? What if the resigning player had Princed a card that needs a decision (say Rebuild, or even a Masquerade)?

The idea sounds appealing, because it doesn't count on the bot having any kind of intelligence near a human, but sometimes decisions must be made. Maybe these cases are so few and far between that the effect on the overall game rankings is tolerable.

Donald X.

Quote from: AdamH on 01 June 2017, 09:31:30 PM
"tl;dr": We don't include bot games on the 2P leaderboard -- what are the reasons for that, and why should they be different than 3P games?
Agreed. Bots are extra-exploitable opponents, so they shouldn't be ranked, so we still shouldn't rank them in multiplayer.

For the joy of finishing out the game, the best option if someone drops is to replace them with a bot. The bot may decide the game between the other two players due to its decisions; there's no avoiding that, it's in the nature of 3-player games. You could have the player drop out completely - or have a bot that just passed - but that's not finishing a 3-player game, that's playing a 3-player game that turned into a messed-up 2-player game. I have played that game! Having a different player (in this case a bot) take over is better.

For ratings, you each beat the person who left, and we can no longer fairly evaluate you vs. the person who stayed; the ratings part of the game is over. The important thing is to not make people unhappy here, rather than to try to get all the data we can. You scored a win vs. someone and then played out the rest of the turns for fun. If you concede too at that point, well okay, you didn't play out the rest of the turns for fun.

JW

Quote from: Donald X. on 02 June 2017, 07:43:04 PM
For the joy of finishing out the game, the best option if someone drops is to replace them with a bot. The bot may decide the game between the other two players due to its decisions; there's no avoiding that, it's in the nature of 3-player games. You could have the player drop out completely - or have a bot that just passed - but that's not finishing a 3-player game, that's playing a 3-player game that turned into a messed-up 2-player game. I have played that game! Having a different player (in this case a bot) take over is better.

For ratings, you each beat the person who left, and we can no longer fairly evaluate you vs. the person who stayed; the ratings part of the game is over. The important thing is to not make people unhappy here, rather than to try to get all the data we can. You scored a win vs. someone and then played out the rest of the turns for fun. If you concede too at that point, well okay, you didn't play out the rest of the turns for fun.

And if it's a 3-player game and both other players resign, there should be an option for the remaining player to continue against two bots, just for the fun of finishing the game.