Villager Workstation Glitch

So ive created a villager hall, and villagers seem to take a work station when i leave the world or they leave render distance, that is not one ive assigned for them. Ive encased them with blocks and somehow they still sense the workstation. (This is on realms)