Items delete when I rejoin

When I rejoin into my survival world as the second or third player(XBox 1), which I already have progress on, I have nothing in my inventory and spawn at the world spawn point. Or, I spawn somewhere I was before but I wasn't there when I left last. It seems it takes away our progress and happened multiple times. Our Xbox still have a lot of storage, so I do not know why this happens. Also, please DO NOT mark as resolved until I find a working solution.