Game takes 20 minutes to load and when I join it kicks me

when I click into the game it takes over 15 minutes to load onto the menu then takes another 15 minutes to load onto the game this has been happening for a while now but none of my friends seem to have this issue so I'm not sure what to do and also some times when I wait 15 minutes for it to load it freezes and I have to reset my whole Xbox if you could help me that would be great! Thank you