Realms won't load with a resource pack

When I try joining a realm with a resource pack or required resource pack it gets stuck on Building terrain and it won't load!

Sometimes it keeps stuck on Building terrain without having any resource packs applied!