Removed scoreboards appear again after a relog

I had some old scoreboards from older versions of the game which I removed & I added some new ones. Every time I enter the world again, the new scoreboards are gone & the old ones reappeared.

I believe I was having this issue in previous snapshots too.

There are no command blocks/functions that are adding/removing the scoreboards