Villagers are despawning

I go out of the village for 5 minutes to get wood and half of the villagers are gone, and you can see them despawn. This also happens when I go on and it lags out 10 seconds later and the villagers disappear at random.