Nether portal crashes realm

The game disconnects very often when I use a portal in nether to back to the overworld. Either it disconnects directly before the overworld is shown/loaded or some time after when I have had time to move around in the overworld. When I reconnect to the Realm an old backup has been used to restore the world. It seems like it is always a backup from the overworld that is used, before I entered the Nether even if I have been in the Nether for quite some time.

This happens very frequently. It seems to happen more when I tried to add more portals. I removed those again to make it playable. In some cases it was 100% reproduce-able when I enabled and used certain portals.