Rating calculation of sigma

[markus]

I think that the updating of sigma is not in line with the Glicko-2 algorithm:
It seems that everyone who played 1 day has sigma=0.0576 and those who played on both days have sigma=0.0554.

The correct values should be much closer to 0.06. If such a jump happens in the Glicko-2 algorithm, it's only because some players idiosyncratic results are that extreme. Therefore, it's virtually impossible that all players jump by so much, but to the (almost?) exact same value.

From a mathematical perspective, what step 5 in the algorithm does, is to find the zero point of the f(x) function on top of page 3. Something seems to go wrong there. In particular the solution for x is significantly smaller than a, so the first ratio must be more negative than it should be.

My hunch is that a correct implementation of sigma will be relatively boring with most players sticking close to 0.06. If there's some downward bias, however, it will also affect phi, which will become smaller than it should be - especially once everyone's phi has become smaller due to playing such that sigma matters more in step 6.

[Stef]

I tried to be careful but of course it's possible there is a mistake. The easiest way to spot that would be... to just post the code here?

Code Select Expand


package matching.ratings;

import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;

public class RatingCalculator {

    public static final double DEFAULT_TOLERANCE = 0.000001;
    public static final double DEFAULT_TAU = 0.42;

    private final double TOLERANCE;
    private final double TAU;


    public RatingCalculator(double TOLERANCE, double TAU) {
        this.TOLERANCE = TOLERANCE;
        this.TAU = TAU;
    }


    public RatingCalculator(double TAU) {
        this.TOLERANCE = DEFAULT_TOLERANCE;
        this.TAU = TAU;
    }


    public RatingCalculator() {
        TAU = DEFAULT_TAU;
        TOLERANCE = DEFAULT_TOLERANCE;
    }


    public double g(double phi) {
        double tmp = 1 + ((3 * phi * phi) / (Math.PI * Math.PI));
        return 1 / Math.sqrt(tmp);
    }


    public double E(double mu, double muj, double phij) {
        double tmp = -g(phij) * (mu - muj);
        return 1 / (1 + Math.exp(tmp));
    }


    public double v(Rating oldRating, ImmutableList<Result> results) {
        double sum = results.stream().mapToDouble(r -> {
            double g = g(r.getOpposition().getDeviation());
            double E = E(oldRating.getSkill(), r.getOpposition().getSkill(), r.getOpposition().getDeviation());
            return g * g * E * (1 - E);
        }).sum();
        return 1 / sum;
    }


    public double d(double v, Rating oldRating, ImmutableList<Result> results) {
        return v * sumResults(oldRating, results);
    }


    public double sumResults(Rating oldRating, ImmutableList<Result> results) {
        return results.stream().mapToDouble(r -> {
            double E = E(oldRating.getSkill(), r.getOpposition().getSkill(), r.getOpposition().getDeviation());
            return g(r.getOpposition().getDeviation()) * (r.getScore() - E);
        }).sum();
    }


    public double f(double x, Rating oldRating, double v, double d) {
        double phi2 = phi2(oldRating);
        double tolerance = Math.pow(TOLERANCE, x);

        double leftNumerator = tolerance * (d * d - phi2 - v - tolerance);
        double leftDenominator = 2 * Math.pow(phi2 + v + tolerance, 2);
        double left = leftNumerator / leftDenominator;

        double rightNumerator = x - a(oldRating);
        double rightDenominator = TAU * TAU;
        double right = rightNumerator / rightDenominator;

        return left - right;
    }

    public double phi2(Rating oldRating) {
        return oldRating.getDeviation() * oldRating.getDeviation();
    }

    public double a(Rating oldRating) {
        return Math.log(oldRating.getVolatility() * oldRating.getVolatility());
    }


    public double newVolatility(Rating oldRating, double v, double d) {
        double phi2 = phi2(oldRating);
        double A = a(oldRating);
        double B;
        if (d * d > phi2 + v) {
            B = Math.log(d * d - phi2 - v);
        } else {
            int k = 1;
            while (f(A - (k * TAU), oldRating, v, d) < 0) k++;
            B = A - (k * TAU);
        }
        double fA = f(A, oldRating, v, d);
        double fB = f(B, oldRating, v, d);

        while (Math.abs(B - A) > TOLERANCE) {
            double C = A + (A - B) * fA / (fB - fA);
            double fC = f(C, oldRating, v, d);
            if (fC * fB < 0) {
                A = B;
                fA = fB;
            } else {
                fA /= 2;
            }
            B = C;
            fB = fC;
        }

        return Math.exp(A / 2);
    }



    public Rating updateRating(Rating oldRating, ImmutableList<Result> results) {
        double v = v(oldRating, results);
        double d = d(v, oldRating, results);
        double newVolatility = newVolatility(oldRating, v, d);
        double phiStar = Math.sqrt(phi2(oldRating) + newVolatility * newVolatility);

        double tmp = 1 / (phiStar * phiStar) + 1 / v(oldRating, results);
        double newDeviation = 1 / Math.sqrt(tmp);

        double newD2 = newDeviation * newDeviation;
        double newSkill = oldRating.getSkill() + newD2 * sumResults(oldRating, results);
        return Rating.builder()
                .setDeviation(newDeviation)
                .setSkill(newSkill)
                .setVolatility(newVolatility)
                .setNumberOfResults(oldRating.getNumberOfResults() + countGames(results))
                .build();
    }


    private int countGames(ImmutableList<Result> results) {
        return results.stream().map(Result::getGameId).collect(ImmutableSet.toImmutableSet()).size();
    }


}


[markus]

Thanks, I think I found the mistake. You're using eps^x in the calcultion of f(x), but it should be e^x.

Code Select Expand

double tolerance = Math.pow(TOLERANCE, x);
should be

Code Select Expand

double tolerance = Math.exp(x);
and you probably shouldn't call it tolerance, because the tolerance is really only used for finding the 0 of this function. The interpretation of exp(x) is actually the new value of sigma^2.

If you fix that and just set all players' sigma to 0.06 again, it should be fine.

[Stef]

If you fix that and just set all players' sigma to 0.06 again, it should be fine.

Ok thanks I fixed it. Will be released tomorrow (most likely).

[markus]

Thanks.

By the way, you could relatively easily calculate updated intraday rankings as well after every game played -  if that is something you want. You would take oldRating from last 0:00 and calculate intradayRating using all games a player has played since 0:00. This coincides with the newRating at next 0:00, if the player doesn't play anymore afterwards. At 0:00 you would exactly do what you do now to calculate newRating (i.e. you don't need the intradayRating) and that would also update the rankings of those that haven't played. This gives you the same rating as now (at 0:00).

A disadvantage would be: intraday, if I play against someone who has played on that day before, I see a different mu and phi than the one that will actually be used for my rating calculation, which would be the one from last 0:00. I don't think that many people would notice that - at least fewer than the ones that ask why it's not updating.  ;)

[SirDagen]

Woohooo. Man, if this is all correct, this is why I love this community. All supportive, dedicated and aiming for the best. Of course there is also the opposite you will face, but if you can focus, it is so beautiful how people support this game.