People cant join me

people cant join me for some reason unless their on my xbox signed in but I'm able to log into my worlds tho and i can invite but like i said there not able to join me it tells them unable to connect