It will not load when you try to connect to big worlds

When my friend tries to connect with me it will not load. I was thinking it is probably because I have a big world. When I try to connect to his world it will load because his is a lot smaller. If you could please try to fix this. Thanks