Unable to connect to multiplayer worlds

Whenever attempting to join a friends world you are presented with the normal "Connecting to multiplayer" loading screen. Sometimes the moving background freezes and then you are presented with the "unable to connect to world" screen. The problem doesn't seem to be coming from my nor the person i'm playing with internet connection, but the game itself