Nether Portal changes location everytime I go through

Previously linked portals keep sending me out to a different location each time I enter it. The first few times werent a big deal, I just built back to my home portal, however it is happening frequently. It is not just one portal, I went through a portal I linked to the end fortress and it spit me out in the nether 2500 blocks away. Furthermore, I also went through the end portal for the first time on that world and fell through the world and died immediately when spawning in. The only way I find I can prevent this from happening is breaking the portal before I want to go back through.