Cartographer Villager Freezing (caused by OptiFine)

When I trade with a cartographer Villager and he levels up to level three, he will freeze and cause the game to also freeze. The only way to fix it is to go into task manager and end Minecraft altogether. Then when I go back into the world it sets me back a bit because I couldn't save.