Unable to connect to world

My Minecraft Pe Xbox live account is working, but every time my friend invites me, it loads for awhile and says "unable to connect to world" We both tried unfriending and re friending, deleting apps, restarting our phones, but nothing works. Its been like this for a good 4 days now.