Cannot join friends world

Every time I try to join my friends world, it says 'unable to connect to world'. When he tries to do the same, he gets the same result. We both have the same game version and we are not on the beta. Our internet connection works too. We tried inviting each other too, doesn't work either.