Villager attachment to job block

When I place many villagers with job blocks in a row then break one job block and place it again, another villager far away, who has a job block and is standing very near to it, attaches to that job block far away from him that I broke it and placed it again. Fix this unlogic bug. Make villagers attach only to job blocks that are very near next them.
Phone: Huawei Mate 20 Lite
Player name: UndoingVessel46