villagers disappearing

I have been playing in my survival world for about a month and there have been two incidents where every single villager in a village disappears. This seems to happen once I turn away from the villagers that they disappear because once I turn back around they are all gone.