You are not white-listed on this server!

A friend of mine is hosting a server for our friend group and it seems I am the only person having this problem. Every week or so, it says you are not white-listed on this server. i have tried multiple ways of trying to fix this but the only thing that seemed to work was leaving and re-joining the realm. I don't want to go through the process of leaving and re-joining the realm every week so any solution would be welcome. I've looked around and quite a few other people have been having the same or similar problems in current and past versions.