If you kick out a switch friend from your world, Other switch friends can no longer join your world

Summary:
If you kick a switch friend out from your world, the other switch friends can no longer join your world. Microsoft account friends aren't affected by this.

Steps to Reproduce:

1. Have a second player join your world
2. kick the player out
3. Try to have a third player join your world

Expected Results:
The player join your world with no problem

Observed Results:
The player can't join, and get "You are not invited to play on this server"

Note:
If you reload the world, the bug won't happen.