Villagers claiming and working at workstations that have been removed

I had about 8 smithing tables with normal villagers using them. those villagers then got converted into zombies and burnt, so I removed the workstations. now new villagers are claiming smithing tables in that location, although they do not physically exist.