Villagers keep exchanging work stations

I have a lot of librarian villagers and they keep exchanging work station. I give them the work station they claim and a few hours later they will stop using them and claim the work station of another villager, so I have to figure out which one they claim and change it out and even if I do that, a few hours later they stop claiming that one and choose a new one!