Trading UI automatically closes when villager wanders away

During the trading, villagers seem to wander away or be pushed away from the player. This may cause the trading interface to close unexpectedly. Since trading involves left mouse click, I sometimes accidentally hit the villagers and then get killed by the angry iron golems.

I hope the trading UI can be maintained even if the villager wanders away.