For a bit of background,
1) I don't think they have your IP. Everything is server based in the game, we can't have client A directly communicate with client B, because that makes it possible for client A to lie, when hacked: "Yes, I totally draw 3 Platinum with my Smithy! And why are you worried that this happened the 10th time in a row?" Even the chat goes to the server, because when spectating there are options to remove spectator chat from what the players see.
But I didn't make the game, so there could be something that exposes your IP to an opponent.
2) Even if they have the IP, they would need access to your ISP's logs to find out who you are. The ability for the FBI in crime shows to instantly pin down where you live when they get your IP is most likely highly fictional. Your opponent does not even have those resources.
That's a simple threat made by bullies to make you feel uncomfortable, because there is just enough credibility that someone might think it possible. All the cases of SWATing I've read about involve the SWATer getting personal information from the victim via chat first (if somebody has your name and home town, the threat to find out where you live is much more real).