villager despawn after leaving simulation/rendor distance

I made a villager trading hall and all was good, i was there for an hour and left the area as I needed my ender chest. then i come to find my villagers disappeared. Also I have watched people like grians video and he keeps on repopulating the village but then the villagers disappear again and again. This NEEDS to be fixed ASAP.