Banner markers reappear on map after banner destruction

Adding markers to maps via banners works well.

After I destroy a banner that had a marker on a map, the marker usually disappears from the map. After a while (I assume reloading the game or leaving the chunk), the marker will be on the map again, although the banner does not exist anymore.

If I get into the chunk of the former banner again or come close to its former location, the map icon disappears again. But it will reappear later on.