My villagers despawn

I created I house full of beds and villagers and gave them 4 stacks of carrots so they can start breeding. After that I went away to get some wood not that far from the house, and when I came back all my villagers were gone and only one baby zombie villager was left. all of this happen during day time and the house wasn't dark it have a decent amount of torches.