I'm not able to join my friends online worlds.

I can't Join my friends worlds. If they have a texture pack, I'll download it, but it will just keep me in the downloading screen, without actually downloading anything. If they don't have a texture pack, I'll join, but it will keep me in the loading screen instead. My friends can join my worlds just fine, but I can't join them. Does anyone have an answer to why I can't join worlds?