Certain villagers refuse to go to the nether

I've been trying to move villagers through nether portals (using mine carts) but certain villagers don't seem to go through no matter what I try. I break the mine cart and the villager hits the portal, the villager freezes (in mid air) and goes back to doing his thing. This only seems to happen to certain villagers as I was able to manage to get a couple in but for the vast majority of them it doesn't seem to work at all.