Server crashes when entering Nether after having removed a Nether Portal

Our world has several portals to the Nether, at some stage I've removed a Nether Portal in the Overworld and moved it to a different spot (moved it roughly by 20 blocks). Now when I enter the Nether Portal on the Nether side to return to the overworld, the server crashes.