I get kicked out of a crossplatform multiplayer world

Whenever I play cross platform whether that be xbox one to minecraft pe or just minecraft pe to minecraft pe I will load into the multiplayer world and after around 1-5 mins i will be kicked off the world and back to the home page, I will then be unable to log back into the world until both I and the person i am playing with restarts their app. This same thing happens to them aswell if they join my world