portal spawns in different place when going BACK to overworld

While speed running and making a quick portal with a lava pool and a water bucket and go trough. after a while when going back my portal spawned underground and not even near my old portal.