Scoreboards modification not saved after closing world

When you modify a scoreboard and then exit the world, and then enter the world again, the scoreboards are not saved and it uses the previous session of scoreboards (from when it last saved).

This is a world which was generated in an earlier version of the game (1.12 I believe) and then opened in the latest snapshot and used with scoreboards.