Can't join friends world and can't get invites

I live in Texas and i can join a few friends in Texas but I can't join on my friends in Utah. This summer I went to Utah and I was right next to my friend on the same network and it still did not work. I was able to join him once but after a tried re joining it did not let me in. I am signed in to my Microsoft account and I have my friend as favorite he has the same for me, the only difference is that I have the Xbox app and he doesn't.

 

Then when he invited me in his minecraft world I got no notification telling me that I have been invited.