You are not white listed

My friend invited to a Minecraft realm and after a day or so it said I was not "Whitelisted" and I was confused, he got someone to add me back in and everyone in the server knows I'm supposed to be there. We thought it would be a one and done deal, but now I can't get on the server at all without being added back onto the server. With my friend being in a different time zone it makes it hard to be on at the same time.