Villager's trades keep changing

I had a librarian villager that traded Fortune III and a few minutes later it just traded papers and bookshelves. I didn't destroy its workstation, just placing down more for other villagers. I tested it on an other map and it looked like they change trades when they restock. It only happens if the player hasn't traded with them yet.