They key to this was Innovation, not the Swap (can also happen with City-state).
Your opponent gained a Skirmisher. As a "When gain" trigger, they played it with Innovation. So now they have a Skirmisher resolving, which reads in part
"This turn, when you gain an Attack card, each other player discards down to 3 cards in hand."
Ok, Skirmisher is done resolving, the game is still looking for potential "When gain" triggers (like they could play a Sheepdog or something like that).
The game finds one, the one just created by Skirmisher! So it makes you discard down to 3 cards in hand.
This feels counter intuitive to many players, but ultimately goes back to very old rulings that for an attack you could first respond with Secret Chamber, a 1st edition Intrigue card that allowed you to draw 2 cards when attacked, then with a Moat you drew from that Secret Chamber.
In Dominion, if you are in the "time window" when you can play Reactions or use triggered abilities for a certain event, newly created such possibilities can and have to be handled at some time. This includes the original example of Secret Chamber drawing Moat, and "new" ones of cards being gained, then played via Innovation, triggering off themselves.
The first widely discussed example of this was Livery + Innovation, your example is similar.