Weird friend connection problem

I can't play with 2 of my friends in a multiplayer world since somehow both of them can join my server but I don't own the multiplayer world and the other one can't join both, it's a very weird relation about the joining, sometimes can't even receive the invitation (xbox)