Villagers are disappearing and now we can't join the realm.

Yesterday I finished my trading area for villagers (I cured random ones from wilderness to get them if that matters). Today, one suddenly vanished. Replace him, go get EXP from a farm, come back, BOOM, the only one present is the replacement guy. We use a backup to see if that helps, only to be stuck on "Joining Realm, this may take a few moments".