When travelling through a portal chunks don't load until I re-start the world.

When I travel through any portal (nether or end) I get transported to the new dimension, but the chunks won't load. I have to quit and restart the world to get the chunks to load. This only happens when I use portals over 100,000 blocks away from (0,0). Portals work fine near original world spawn.