When you join worlds they say unable to connect to world

Hello, when I try to play with my friend it sometimes says "unable to connect to worldworld." The thing is a VPN fixes this, and then that stops working, though without it I can play then. I don't want to keep switching from VPN to no VPN, please fix this issue.