Cannot join other players and vise versa

For some reason whenever I try to join someone else I keep getting kicked out, it is saying that you are not invited to this server or you were kicked, and the same goes for when people try to join me...