Villagers Take Job Despite Being Many Blocks Away From Job Block

Villager is in it's own room underground and no way to path find up to any job blocks but he still links to job block above ground when there is literally a job block right in front of him in the same room. Isn't there  This is very annoying as it makes villagers very hard to deal with when creating trading areas.