Villagers despawning after cured with potion and golden apple

After I turn a zombie villager back into a normal villager, it despawns a couple minutes later. This happened to me twice. I was changing the difficulty level now and then to restore my health, but i don't know if that triggered it. The villagers also don't run away from zombies or go inside at night time.