Shift clicking items out of output slot of villager can cause the villager to not take the items

The bug

If you shift click items out of the output slot, the villager sometimes won't take the item.
This happens most of the time with emeralds, and gold. However, this can also happen with common items like wheat, ect. Also seems like if the villager reaches an certain profession level, this will be fixed by itself.

How to reproduce

1. Get a stack of emeralds and summon an Armorer villager.

   /summon minecraft:villager ~ ~ ~ {VillagerData:{profession:"armorer","type":"plains"}}
   /give @s emerald 64
   

2. Trade for the iron helmet, shift click them out of the output slot.
   → The villager didn't take any emerald but still gave you the iron helmet