Villager Changes Profession After Trading/Leveling

I had an Armorer that I had traded with enough to raise its level to 3. I broke its Blast Furnace, thinking another villager was linked to that particular workstation. Since it was my leveled Armorer that was linked to it, it unlinked when I broke the Blast Furnace. As soon as it unlinked from the Blast Furnace, it linked to a Barrel that I was used to feeding Furnace fuel into a Hopper which caused it to become a Fisherman. I broke the Barrel, but the villager remained a Fisherman. Even after breaking the Barrel and placing the Blast Furnace back down, the villager still remained a Fisherman.