Villagers are disappearing

When I went so far away from my villagers all my farmers despawned. Another time that they disappeared is I saved and quit my world and the next time I went back in they were all gone, they just disappeared all of them and it was not on a realm. My villagers were safe they were in a 3 block tall box with torches and no entrances besides ladders, that shouldn't matter because I saw them before I got off and when I got on they weren't there so there is a glitch