Villagers Disappearing

Put the summary of the bug you're having here

What I expected to happen was...:

Villagers not despawning while in world

What actually happened was...:
Villagers will slowly disappear from a village after certain number of days

Steps to Reproduce:
1. Create a new world
2. Find a village
3. Stay around the area and see villager population decrease