bug with /scoreboard

Serious bug with the /scoreboard command. Instead of a player leaving, then the scoreboard registering them as "Player Offline" and then it saves when they log back in, it keeps making new versions of "Player Offline" that I cannot seem to reset the scores of.