My villagers are stealing me (Taking more paper than displayed)

When you trade with a high ranked librarian, they display like "23 paper for 1 emerald" and when you do the trade, they take more than that (see the clip).

Note: The amount they steal seems to vary with rank.

Clip notes: Some people told me to wait a bit to see if I don't get my paper back. It nerver did.

Some villagers seem to be unaffected by this issue.