Multiplayer issue with one friend

so basically it doesn’t male me enter the world of my friend. And this only happens with me and him, because with another friend there isn’t this problem. We tried to invite eachother by ps4 account and Microsoft account, but nothing. Could you please resolve this problem?