Unable to Connect to World

I continue to attempt to have my friends join my game they it seems that even though I invite different friends they all seem to get the message "Unable to connect to world" I have another friend who has the same issue when he tries to host. Yet a third friend is able to host and have both of us join with no issues.  I saw a different and older post where this issue was marked as resolved but it clearly is not fixed. I am beyond frustrated with this as all I'm trying to do is play a game with some friends. I have checked my Xbox privacy setting but that seems to do nothing and has changed nothing with my connections. Someone please help.