Can't join friends, and friends can't join me (Private Worlds)

I can't join any worlds hosted by my friends on XBOX, and none of my friends are able to join me, there is no error code, just says "Unable to connect to world" or "Disconnected from Server".