Overworld despawn from the Nether

If I make a Nether portal underground, enter it, then return, I spawn at a random place.

The first time I experienced this was a year or two ago. I came back to the overworld literally miles and miles away from where I was supposed to be. I didn't even spawn in the portal in the small cave under the sea, but on the sea. I was racing against time to get my stuff back.

Lately it's happened again but I was much luckier. I spawned no way near as far from my base. This time it was a snowy biome, not how it's supposed to be (underground where the original portal is). I thought I'd remove the portal (in the snowy biome) and try again. Though I still spawned in at the same place.

I never had an issue in the nether. I have a portal not as far underground, but in the floor leveled short hill. Spawning back into the overworld I remember it's exactly how it should be. So it's just the overworld underground nether portals that has this return spawn error.