When someone would try to join my worlds that they're stuck on the locating server screen for a very long time.

I have an issue when someone would try to join my worlds that they're stuck on the locating server screen for a very long time. I have the multiplayer on and I have it on where everyone can join the world without an invite to join it. It even happens if someone joins using a different console/platform. This has been going on for awhile and it has gotten kind of annoying, to be honest. I just hope you guys could figure it out and hopefully be able to fix this problem.