So apparently, when I actually get a huge winning streak right from the start, the game decides that I should get disconnected from a few matches afterwards. And now you know why people want Nintendo to actually use their servers for the actual matches and not just for the matchmaking.