Disappearing villagers

Whenever I leave my village in order to do some exploration or building, and I don't trade or breed my villagers for about 2-3 consecutive in game days, they suddenly disappear. I've tried blocking them in houses and trapping in 2x1 pits but that doesn't work either. The entire area is mob proof and there are no rivers, caves or cliffs. It happens in any world I play. Please fix this, I've been losing important librarians and emerald fletchers.