That's correct. Over the years, some players have abused the system to have rated games with the kingdom taylored to some degree or other (required Platinum, forced a certain card combo, etc.). That lead to them going up the leaderboard, when it turned out all they ever played was the same or very similar kingdoms.
So sometime at or after Allies was released and the UI had to be adapated, the current behaviour was introduced, where any change to influence what cards appear in the kingdom leads to an unrated game.
