Villagers disappearing in single player when going through a nether portal

I need to put 5 villagers into a 1x3 area for an iron farm, but every time I manage to do that, when I go into the nether to get more, some of them disappear from the pit. It is very difficult to move villagers where you want, And I just want this fixed. It is the most frustrating thing I have ever done just to have it disappear instantly