Game Crash when Realm Host uploads Map from another Host, and that Host Joins!

My friends and I created a large Realm. The file is approx. 550mb. The original host downloaded the Realm to their PC and shared it with me via dropbox. I uploaded this World and hosted it on my Realm. Everything responds fine except for when the old host joins the game when I'm online or if I join when they are online, then the game crashes immediately to desktop with no warnings at all. Sometimes the game can operate a minute or two, sometimes it cant even reload if the old host is still online in my Realm. If I try to rejoin, it doesn't just crash me, it crashes us both.