Friends Aren't Getting Invites

So as of the lately I have been trying to invite friends to my world to play Minecraft on the Xbox One. I try inviting friends, but they seem to never get the invites although I am pressing the invite button to invite them. This only happens on Minecraft, because every other game they get invites to.