Villagers being pushed/stuck on exp orbs

In my villager trading hall, whenever i trade with a villager, the xp orbs makes the villager stuck and is pushed away by the orbs, only way i fixed it is when i go in where the villager is and take the xp, which makes the villager unstuck.